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WELCOME  TO  THE  2018  HUMAN 
SYSTEMS  CONFERENCE 


Welcome.  The  2018  Conference  of  the  NDIA  Human  Systems 
Division  (HSD)  is  designed  to  serve  the  Department  of  Defense 
and  strengthen  the  defense  industry.  This  is  the  mission  of 
the  NDIA  and  its  members:  1 ,600  corporations  and  85,000 
individuals.  The  HSD  conference  supports  the  mission  by 
bringing  DoD  S&T  leadership,  representatives  of  industry,  and 
other  guests  into  discussion  over  accomplishments  of  DoD 
human  systems  research  and  development,  gaps  in  DoD 
human  systems,  and  emerging  research  and  technology  that 
can  bridge  those  gaps.  This  conference  is  a  small  and  vital 
forum  for  all  of  us  to  learn,  to  envision  new  solutions,  and  to 
build  the  relationships  and  opportunities  to  collaborate. 

The  NDIA  Human  Systems  Division  is  chartered  to  support  the 
DoD  Human  Systems  Community  of  Interest  (HS  COI).  Our 
subcommittee  structure  (represented  by  the  list  of  technical 
sessions  in  this  program)  is  identical  that  of  the  HS  COI,  with 
the  exception  of  one  addition:  this  Division  has  established 
a  subcommittee  on  Human  Systems  Metrics  to  help  assess 
and  promote  the  value  of  human  systems  research  and 
development,  techniques,  and  technologies.  The  leaders  of 
our  five  subcommittees  plan  this  conference,  participate  in 


studies  for  DoD,  and  grow  the  Human  Systems  community.  I 
urge  you  to  join  one  of  the  Division’s  subcommittees.  To  pursue 
that  opportunity,  contact  NDIA,  or  talk  to  any  of  the  Human 
Systems  Division  subcommittee  co-chairs  at  this  conference. 

Thanks  to  the  team  that  organized  our  conference:  the 
conference  co-chairs,  LCDR  Jacob  N.  Norris,  Ph.D.,  and 
Eric  Jones;  their  team  members  on  the  HSD  sub-committee 
co-chairs;  and  the  professional  staff  of  NDIA.  Thanks  also  to 
the  representatives  of  the  DoD  Human  Systems  Community 
of  Interest,  who  have  joined  us  at  this  event.  A  special  note 
of  appreciation  to  our  sponsors:  Ball  Aerospace,  Rockwell 
Collins,  and  DCS  Corporation.  Finally,  welcome  to  all  of  you 
who  have  come  here  to  understand,  improve,  and  accelerate 
the  research  and  development  of  better  human  systems  for  the 
Department  of  Defense. 


Respectfully, 


4T 

Jared  Freeman,  Ph.D. 

Chair,  NDIA  Human  Systems  Division 
Chief  Scientist,  Aptima,  Inc. 


SCHEDULE  AT  A  GLANCE 


TUESDAY,  MARCH  13 

General  Session 

Singleton/Miller  Ballroom 
8:00  am  -  5:00  pm 

Networking  Poster  and  Demonstration  Session 

von  Sternberg/Hazel  Ballroom 

1:15-3:15  pm 

Networking  Reception 

Waterford  Foyer 
5:00  -  6:30  pm 


WEDNESDAY,  MARCH  14 

General  Session 

Singleton/Miller  Ballroom 
8:00  am  -  5:00  pm 

Round  Table  Discussion 

Singleton/Miller  Ballroom 
3:15  -  4:45  pm 
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WHO  WE  ARE 

The  National  Defense  Industrial  Association  is  the  trusted  leader  in 
defense  and  national  security  associations.  As  a  501(c)(3)  corporate 
and  individual  membership  association,  NDIA  engages  thoughtful  and 
innovative  leaders  to  exchange  ideas,  information,  and  capabilities  that 
lead  to  the  development  of  the  best  policies,  practices,  products,  and 
technologies  to  ensure  the  safety  and  security  of  our  nation.  NDIA’s 
membership  embodies  the  full  spectrum  of  corporate,  government, 
academic,  and  individual  stakeholders  who  form  a  vigorous,  responsive, 
and  collaborative  community  in  support  of  defense  and  national  security. 
For  more  information,  visit  NDIA.org 


HUMAN  SYSTEMS 
DIVISION 


LEADERSHIP  AND 
COMMITTEES 

Dr.  Jared  Freeman _ 

Human  Systems  Chair 
Chief  Scientist,  Aptima,  Inc. 

BG  Peter  Palmer,  USA  (Ret) 

Human  Systems  Vice  Chair 
President,  P2  Consulting  Services 

Scott  Kozak 

Human  Systems  Deputy  Chair 
Managing  Director,  CogniSens 
Applied  Research  Center 


WHO  WE  ARE 


NDIA’s  Human  System  Division  promotes  the  exchange  of  technical  information 
and  discussions  between  government,  industry  and  academia,  and  the  expansion 
of  research  and  development  in  areas  related  to  the  human  as  a  system  whose 
performance  must  be  integrated  into  any  military  system  of  systems.  To  this  end,  the 

division  will  serve  as  an  infrastructure  by  providing  a  variety  of  ways  for  government,  industry  and  academia  to  collaborate 
to  advance  human  performance  in  air,  land,  see,  space  and  cyberspace  through  research,  education  and  consultation. 
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EVENT  INFORMATION 


EVENT  WEBSITE 


EVENT  CONTACT 


EVENT  THEME 


CONFERENCE 

LEADERSHIP 


SESSION  CHAIRS 


SURVEY  AND 
PARTICIPANT  LIST 


HARASSMENT 

STATEMENT 
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Andrea  Lane 

Meeting  Planner 
(703)  247-2554 
alane@ndia.org 


Human  Systems  in  Emerging  Domains:  Autonomy,  Human  Augmentation  and  Cyber 


Eric  Jones 

Industry  Chair 


LCDR  Jake  Norris,  USN 

Government  Chair 


Brad  Chedister 

Industry  Co-Chair,  PS&WP 

Glenn  Gunzelmann 

Government  Co-Chair,  PAE&T 

Dr.  Kelly  Hale 

Industry  Lead,  HSM 


Dr.  James  McCarthy 

Industry  Co-Chair,  PAE&T 

Dr.  Todd  Nelson 

Government  Co-Chair,  SI&CP 

CDR  Henry  Phillips,  USN 

Government  Co-Chair, 
HI3Lead 


George  Salazar 

Government  Lead,  HSM 

Peter  Squire 

Government  Co-Chair,  PS&WP 


A  survey  and  list  of  attendees  (name  and  organization  only)  will  be  emailed  to  you  after  the 
conference.  NDIA  would  appreciate  your  time  in  completing  the  survey  to  help  make  our 
event  even  more  successful  in  the  future. 


NDIA  is  committed  to  providing  a  professional  environment  free  from  physical,  psychological 
and  verbal  harassment.  NDIA  will  not  tolerate  harassment  of  any  kind,  including  but  not 
limited  to  harassment  based  on  ethnicity,  religion,  disability,  physical  appearance,  gender,  or 
sexual  orientation.  This  policy  applies  to  all  participants  and  attendees  at  NDIA  conferences, 
meetings  and  events.  Harassment  includes  offensive  gestures  and  verbal  comments, 
deliberate  intimidation,  stalking,  following,  inappropriate  photography  and  recording, 
sustained  disruption  of  talks  or  other  events,  inappropriate  physical  contact,  and  unwelcome 
attention.  Participants  requested  to  cease  harassing  behavior  are  expected  to  comply 
immediately,  and  failure  will  serve  as  grounds  for  revoking  access  to  the  NDIA  event. 


AGENDA 
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7:00  am  -  6:00  pm 

REGISTRATION 

VON  STERNBERG/HAZEL  FOYER 

7:00  -  8:00  am 

NETWORKING  BREAKFAST 

MILLER/VON  STERNBERG  FOYER 

8:00  -  8:15  am 

WELCOME  AND  INTRODUCTORY  REMARKS 

SINGLETON/MILLER  BALLROOM 

Dr.  Jared  Freeman 

Human  Systems  Division  Chair 

Chief  Scientist,  Aptima,  Inc. 

8:15  -  9:00  am 

KEYNOTE  ADDRESS 

SINGLETON/MILLER  BALLROOM 

Dr.  Brian  Pierce 

Office  Director,  Information  Innovative  Office  (120) 

Defense  Advanced  Research  Projects  Agency  (DARPA) 

9:00  - 10:30  am 

PANEL:  HUMAN  SYSTEMS  INTERFACE  RESEARCH  AND 
DEVELOPMENT  CHALLENGES  -  GOVERNMENT  PERSPECTIVE 

SINGLETON/MILLER  BALLROOM 

LCDR  Peter  Walker,  USN 

Military  Deputy,  Human  and  Bioengineered  Systems,  Code  341,  Office  of  Naval  Research 

Moderator 

Laurie  Fenstermacher 

Principal  Research  Lead  Open  Source  Analysis,  711  HPW/RHXM 

Glenn  Gunzelmann 

Senior  Research  Psychologist,  Air  Force  Research  Laboratory 

Dr.  Todd  Nelson 

Division  Chief,  711  HPW/AFRL 

Peter  Squire 

Program  Officer,  Office  of  Naval  Research 
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10:30  - 11:00  am 

NETWORKING  BREAK 

MILLER/VON  STERNBERG  FOYER 

11:00  - 11:05  am 

INTRODUCTION  TO  TECHNICAL  SESSION 

SINGLETON/MILLER  BALLROOM 

Eric  Jones 

Conference  Industry  Chair,  NDIA  Human  Systems  Division 

Human  Systems  Architect,  The  Charles  Stark  Draper  Laboratory,  Inc. 

11:05  am  - 12:15  pm 

SESSION  1:  PROTECTION,  SUSTAINMENT  AND  WARFIGHTER 
PERFORMANCE  (PS&WP) 

SINGLETON/MILLER  BALLROOM 

Collaboration  for  Breakthrough  Innovation  in  Human 

Performance  Monitoring  for  the  Warfighter 

Dr.  Melissa  Grupen-Shemansky 

SEMI-Flex,  Tech 

FitForce  Planner:  Optimizing  USMC  Physical  Fitness 

Timothy  Clark 

Aptima,  Inc. 

Transitioning  Laboratory  Neuroscience  to  the  Real-World  through 

Deep  Learning:  Using  Evoked  Potentials  to  Assess  State 

Dr.  Stephen  Gordon 

DCS  Corporation 

Optimizing  Future  Soldier  Systems  through  the  Incorporation  of  Human  Aspects 
into  the  Soldier  as  a  System  Domain  using  the  Systems  Modeling  Language 

Sean  Pham 

U.S.  Army  ARDEC 

12:15  - 1:15  pm 

NETWORKING  LUNCH 

VON  STERNBERG/HAZEL  BALLROOM 

1:15  -  3:15  pm 

NETWORKING  POSTER  AND  DEMONSTRATION  SESSION 

VON  STERNBERG/HAZEL  BALLROOM 

3:15  -  3:30  pm 

REFRESHMENT  BREAK 

MILLER/VON  STERNBERG  FOYER 
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3:30-4:40  pm 

SESSION  2:  HUMAN  SYSTEMS  METRICS  (HSM) 

SINGLETON/MILLER  BALLROOM 

Human  Systems  Metrics  Applied  to  Optimize  Warfighter  Capability 

Sarah  Orr 

711  HPW/HP 

Realizing  Our  Collective  Vision  by  2025:  Leveraging  Advances  in 
Artificial  Intelligence  and  Autonomy  with  Human  Systems  Advances  in 
Human-Machine  Symbiosis  to  Realize  Our  Roadmap  to  the  Future 

Dr.  Denise  Nicholson 

Soar  Technology,  Inc. 

Usability  Scorecard 

Deidrick  Capers 

Millennium  Corporation 

Julia  Ruck 

PM  DCGS-A 

HSI  T&E  Methods  and  Metrics  to  Assess  User-Automation  Interaction 

Dr.  Janae  Lockett-Reynolds 

DHS 

4:40  -  5:00  pm 

DAY  1  WRAP  UP 

SINGLETON/MILLER  BALLROOM 

Dr.  Jared  Freeman 

Human  Systems  Division  Chair 

Chief  Scientist,  Aptima,  Inc. 

5:00  -  6:30  pm 

NETWORKING  RECEPTION  (CASH  BAR) 

WATERFORD  FOYER 

7:00  am  -  3:00  pm 

REGISTRATION 

VON  STERNBERG/HAZEL  FOYER 

7:00  -  8:00  am 

NETWORKING  BREAKFAST 

MILLER/VON  STERNBERG  FOYER 


WELCOME  REMARKS 

SINGLETON/MILLER  BALLROOM 

Dr.  Jared  Freeman 

Human  Systems  Division  Chair 
Chief  Scientist,  Aptima,  Inc. 


PLENARY  ADDRESS 

SINGLETON/MILLER  BALLROOM 

Dr.  Paul  Zablocky 

Division  Director,  Complex  Hybrid  Warfare  Sciences  Division,  Expeditionary  Maneuver  Warfare  and  Combating 
Terrorism  Science  and  Technology  Department,  Office  of  Naval  Research 


PANEL:  HUMAN  SYSTEMS  INTEGRATION  METRICS 

SINGLETON/MILLER  BALLROOM 

Dr.  John  Tangney 

Director,  Human  &  Bio-Engineered  Systems,  Office  of  Naval  Research 
Moderator 

Rick  Craft 

Systems  Analyst  &  Principal  Member  of  the  Technical  Staff,  Sandia  National  Laboratory 

Dr.  Kelly  Hale 

Senior  Vice  President,  Technical  Operations,  Design  Interactive,  Inc. 

BG  Peter  Palmer,  USA  (Ret) 

President,  P2  Consulting  Services 

John  Plaga 

Aerospace  Engineer,  711  HPW/HPIF 


NETWORKING  BREAK 

MILLER/VON  STERNBERG  FOYER 


DWIGI  IT  D.  EISEN I  [QWER  AWAE I  > 


*  ■  I'RESENI  El)  TO  JHE  KONORABI.E  SENATOR 

JAMES  1NIIOFE 

■* 

WEDNESDAY,  APRIL  1 3  201 8  |;  fi  30  PM  |  BUSINESS  DKfcSS  ‘ 

r m  rjt*- cars.  row.  pentagon  city 

R  J  G  1ST  It  R  l  OD  A  V  AT  NDiA.ORG/EISENtlOWER 
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10:40  - 11:50  am 


11:50  am  -  12:45  pm 


12:45-1:55  pm 


SESSION  3:  PERSONALIZED  ASSESSMENT, 
EDUCATION  &  TRAINING  (PAE&T) 

SINGLETON/MILLER  BALLROOM 

Virtual  and  Augmented  Reality  for  Training  and  Cognitive  Aides 

Peggy  Wu 

United  Technologies  Research  Center 


Developing  a  Predictive  Model  of  Army  Marksmanship  Performance 

Dr.  Gregory  Goodwin 
U.S.  Army  Research  Laboratory 

Man  vs.  Machine:  Comparing  Traditional  versus  Big  Data 
and  Machine  Learning  to  Predict  Expertise 

Dr.  Krista  Ratwani 

Aptima,  Inc. 


Perceptual-Cognitive  and  Physiological  Assessment  of  Training  Effectiveness 

Amy  Diderksen 

Rockwell  Collins  Simulation  &  Training  Solutions 


NETWORKING  LUNCH 

VON  STERNBERG/HAZEL  BALLROOM 


SESSION  4:  SYSTEMS  INTERFACE  AND  COGNITIVE 
PROCESSING  (SI&CP) 

SINGLETON/MILLER  BALLROOM 

Towards  Natural  Dialogue  with  Robotics:  ARL  Bot  Language 

Dr.  Matthew  Marge 

U.S.  Army  Research  Laboratory 

VR  Technologies  for  Next-Generation  Battle  Management  Command  and  Control 

Dr.  Erin  Cherry 

Northrop  Grumman  Mission  Systems 


Developing  an  Autonomous  Task  Manager  for  Intelligence, 
Surveillance,  and  Reconnaissance  Human-Machine  Teams 

Jennifer  Lopez 

711  HPW/RHXM 

Seeing  into  the  Black  Box:  Using  Eye  Tracking  in  User-Driven 
Workflows  to  Better  Understand  Decision-making  Processes 

Dr.  Kristin  Divis 

Sandia  National  Laboratories 
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1:55  -  3:00  pm 

SESSION  5:  HUMAN  INFORMATION,  INTERPRETATION  AND 
INFLUENCE  (HI3) 

SINGLETON/MILLER  BALLROOM 

Panel:  Operations  in  the  Information  Environment 

Laurie  Fenstermacher 

Principal  Research  Lead  Open  Source  Analysis,  711  HPW/RHXM 

Moderator 

Dr.  Rebecca  Goolsby 

Program  Officer,  Office  of  Naval  Research 

Eric  Wallace 

Chief,  Concepts  &  Requirements,  Joint  Information  Operations  Warfare  Center/Enterprise  Operations  Division 

3:00  -  3:15  pm 

REFRESHMENT  BREAK 

MILLER/VON  STERNBERG  FOYER 

3:15  -  4:45  pm 

ROUNDTABLE  DISCUSSIONS 

SINGLETON/MILLER  BALLROOM 

Protection,  Sustainment  and  Warfighter  Performance  (PS&WP) 

Human  Systems  Metrics  (HSM) 

Personalized  Assessment,  Education  &  Training  (PAE&T) 

Systems  Interface  and  Cognitive  Processing  (SI&CP) 

Human  Information,  Interpretation  and  Influence  (HI3) 

4:45  -  5:00  pm 

CLOSING  REMARKS 

SINGLETON/MILLER  BALLROOM 

Dr.  Jared  Freeman 

Human  Systems  Division  Chair 

Chief  Scientist,  Aptima,  Inc. 

5:00  pm 

CONFERENCE  ADJOURNS 

The  NDIA  has  a  policy  of  strict  compliance  with  federal  and  state  antitrust  laws.  The  antitrust  laws  prohibit  competitors  from  engaging  in  actions  that  could 
result  in  an  unreasonable  restraint  of  trade.  Consequently,  NDIA  members  must  avoid  discussing  certain  topics  when  they  are  together  at  formal  association 
membership,  board,  committee,  and  other  meetings  and  in  informal  contacts  with  other  industry  members:  prices,  fees,  rates,  profit  margins,  or  other  terms 
or  conditions  of  sale  (including  allowances,  credit  terms,  and  warranties);  allocation  of  markets  or  customers  or  division  of  territories;  or  refusals  to  deal  with  or 
boycotts  of  suppliers,  customers  or  other  third  parties,  or  topics  that  may  lead  participants  not  to  deal  with  a  particular  supplier,  customer  or  third  party. 
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BIOGRAPHIES 


DR.  BRIAN  PIERCE 


Office  Director,  Information  Innovation  Office 
Defense  Advanced  Research  Projects  Agency  (DARPA) 


- JR 

Dr.  Brian  Pierce 

■  joined  DARPA  in 

2014  and  serves 

as  the  director  of  the  DARPA  Information 
Innovation  Office  (120). 

Dr.  Pierce  has  30  years  of  experience 
developing  advanced  technologies  in 
the  aerospace/defense  industry.  Prior 
to  joining  DARPA,  he  was  a  technical 


director  in  Space  and  Airborne  Systems 
at  the  Raytheon  Company.  During  his  first 
tour  at  DARPA,  he  served  as  the  deputy 
office  director  of  the  Strategic  Technology 
Office  from  2005  to  2010.  From  2002  to 
2005,  he  was  executive  director  of  the 
Electronics  Division  at  Rockwell  Scientific 
Company  in  Thousand  Oaks,  California. 
From  1983  to  2002,  he  held  various 
engineering  positions  at  Hughes  Aircraft 


Company  and  Raytheon  in  southern 
California. 

Dr.  Pierce  earned  a  Doctor  of  Philosophy 
degree  in  chemistry,  a  Master  of  Science 
degree  in  chemistry  and  a  Bachelor 
of  Science  degree  in  chemistry  and 
mathematics  from  the  University  of 
California  at  Riverside.  He  has  more  than 
20  U.S.  patents. 


DR.  PAUL  ZABLOCKY 

Division  Director,  Complex  Hybrid  Warfare  Sciences  Division,  Expeditionary  Maneuver  Warfare  and 
Combating  Terrorism  Science  and  Technology  Department 
Office  of  Naval  Research 


ETvbJ  j  -  i  I  1 

I  Dr.  Paul  G. 

Zablocky 

currently  serves  at  the  office  of  Naval 
Research  (ONR)  as  the  Division 
Director  of  the  Complex  Hybrid  Warfare 
Sciences  Division  (Code  301)  within 
the  Expeditionary  Maneuver  Warfare 
and  Combating  Terrorism  Science 
and  Technology  Department.  He  is 
responsible  for  leading  and  directing  an 
integrated  portfolio  of  basic  research, 
applied  research,  and  advance 
technology  development  science  and 
technology  (S&T)  efforts  in  support  of  the 
United  States  Marine  Corps  (USMC)  and 
Naval  Special  Warfare. 

Dr.  Zablocky  served  as  the  Director  of  the 
US  Army  Communications  Electronics 
Research  Development  and  Engineering 
Center  (CERDEC)  Intelligence  and 
Information  Warfare  Directorate  (I2WD) 
from  April  2015  to  April  2016  and  as 


the  Director  of  Space  and  Terrestrial 
Communications  Directorate  (S&TCD) 
from  June  2013  to  April  2015.  In  both 
organizations  he  provided  leadership 
and  guidance  to  approximately  800 
government  civilians  and  contractors 
who  executed  S&T  programs,  developed 
prototypes,  and  provided  systems 
engineering  support  for  Army  programs 
in  the  areas  of  communications, 

Electronic  Warfare  (EW),  and  intelligence. 
Prior  to  that  Dr.  Zablocky  served  as 
the  Senior  Research  Scientist  for  EW 
Technology  (ST)  at  CERDEC  I2WD  where 
he  was  responsible  for  developing, 
planning,  coordinating,  and  executing  the 
Army’s  EW  S&T  portfolio.  Paul  started 
his  government  career  in  2005  designing 
and  developing  prototype  EW  systems 
that  transitioned  to  Army  Programs  of 
Record.  He  has  over  30  years  of  research 
and  development  experience  in  Electrical 
Engineering  working  in  both  defense  and 


commercial  industries.  He  served  in  the 
Navy  from  1985  to  1989  achieving  the 
rank  of  Lieutenant. 

Dr.  Zablocky  received  a  Professional 
Master  of  Business  Administration  from 
the  University  of  Massachusetts,  a  Doctor 
of  Philosophy  Electrical  Engineering 
from  the  University  of  Pennsylvania,  a 
Master  of  Science  Electrical  Engineering 
from  the  University  of  Central  Florida 
and  Bachelors  of  Science  in  Electrical 
Engineering  and  Physics  from  Fairleigh 
Dickinson  University. 

He  is  married  to  Barbara.  Their  daughter, 
Amanda,  graduated  from  the  Naval 
Academy  and  is  now  a  pilot  in  Helicopter 
Sea  Combat  Squadron  Eight  assigned 
to  the  USS  Nimitz.  Their  son,  Daniel,  is 
a  recent  graduate  of  The  United  States 
Military  Academy  and  is  in  the  Army 
Basic  Officer  Leader  Course  at  Fort 
Benning,  GA. 


POSTER  &  DEMONSTRATION  SESSION 


Mixed  Reality  Environment  for  the 
Study  of  Human-UAV  Interaction 

Dr.  Zhuming  Ai 

Naval  Research  Lab 


Initial  Evaluation  of  Adaptive 
Language  Learning  Technology 

Dr.  Alan  Carlin 

Aptima,  Inc. 


Dynamic  Robot  Operator  Interface  (DROID) 
Assessment,  Guidance,  and  Engineering  Tool  (AGENT) 

Lisa  Baraniceki 

AnthroTronix,  Inc. 

Preliminary  Job  Task  Analysis  of  a  Cyber  Kill 
Chain  and  its  Application  to  Cyber  Defense 

Doron  Becker 

Department  of  Homeland  Security 

Marianne  Paulsen 

Department  of  Homeland  Security 

The  Effect  of  a  Powered  Lower-Body  Exoskeleton 
on  Physical  and  Cognitive  Warfighter  Performance 

Blake  Bequette 

Massachusetts  Institute  of  Technology 

Facial  Emotional  Expression 
Recognition  Study  (FEERS) 

Gregory  Black 

Electric  Boat  Corporation 


Human  Machine  Interaction  Interface  Display 
Design:  A  Perceptual  Approach  in  Hilbert  Space 

Dr.  Mustafa  Canan 

U.S.  Army  Research  Laboratory 

Impact  of  Torso-borne  Load  Redistribution  on 
Soldier  Biomechanics,  Metabolics,  and  Comfort 

Marina  Carboni 

Natick  Soldier  Research  Development  &  Engineering  Center 


Adaptive  Automation  in  Sensemaking 

Dr.  Daniel  Cassenti 

U.S.  Army  Research  Laboratory 

Interactive  Next-generation  Testbed 
Environment  for  Retention  and  Assessment 
of  Computer-based  Training  (INTERACT) 

Ian  Coffman 

AnthroTronix,  Inc. 


Perceptual-Cognitive  &  Physiological 
Assessment  of  Training  Effectiveness 

Amy  Dideriksen 

Rockwell  Collins  Simulation  &  Training  Solutions 


Cognitive  Operations  Gear  Pack  (COG  Pack™): 

A  Capability  for  Realizing  Real-Time  Cognitive 
Performance  Assessment  in  Air  Force  Operations 

Allen  Dukes 

711  HPW/RHCPA 

Confined  Spaces  Monitoring  System 

Kevin  Durkee 

Aptima,  Inc. 

Wearable  Tactile  Display  for  Hands-Free  Covert 
Communications  with  Semi-Autonomous  Systems 

Dr.  Linda  Elliott 

U.S.  Army  Research  Laboratory 
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Topside  -  Unmanned  System  Command  and  Control 
for  Mission  Planning,  Piloting,  and  Analysis 

Thomas  Fulton 

Naval  Undersea  Warfare  Center  Division  Newport 

Robust  Personalization  of  Training  and 
Assessment  through  the  Generalized 
Intelligent  Framework  for  Tutoring  (GIFT) 

Dr.  Benjamin  Goldberg 

U.S.  Army  Research  Laboratory,  HRED 


Characterizing  Tactical  Decisions  through 
Exploratory  Multivariate  Analysis 

Dr.  Chris  Hale 

Georgia  Tech  Research  Institute 

Analyzing  Expert  Marksmanship  from 
the  Human  Centered  Perspective 

Dr.  Leif  Hasselquist 

Natick  Soldier  Research  Development  &  Engineering  Center 


Training  a  Traditional  High  Risk  Organization 
in  Resilience  Engineering 

Dr.  Christopher  Nemeth 

Applied  Research  Associates,  Inc. 

Personnel  Optimization  for  Human-Machine 
Teaming  in  the  Maritime  Domain 

LCDR  Jacob  Norris,  USN 

SPAWAR  Systems  Center  Pacific 


Intelligent  Humans  Systems  Integration  - 
Could  we  have  helped  prevent  the  McCain 
and  Fitzgerald  Collisions  at  sea? 

Dr.  Nandakumar  Ramanujam 

ASSETT,  Inc. 


Enhancing  Mission  Performance:  Design 
Heuristics  for  Augmented  Reality 

Kimberly  Ryan 

The  Charles  Stark  Draper  Laboratory,  Inc. 


Using  Artificially  Intelligent  Computer  Based 
Training  for  more  Sophisticated  Learning  Needs 

Cenetra  Johnson 

The  George  Washington  University 


Using  Work  Models  to  Derive  Assessment 
Measures  for  Cyber  Protection  Teams 

Dr.  Stoney  Trent 

U.S.  Army  War  College 


Laser  Eye  Protection:  Balancing  Measuring  Coordination  in  Multi- 

Protection  and  Performance  Agent  Reinforcement  Learning 

Dr.  Julie  Lovell  Michael  Walton 

711th  HPW  /  Bioeffects  Division,  Optical  Radiation  Branch  SPAWAR  Systems  Center  Pacific 


Ambient  Activity  Monitors  (AAMs)  to  Display 
Hidden  Computer  System  Information 

Dr.  Jamie  Lukos 

SPAWAR  Systems  Center  Pacific 
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THANK  YOU  TO  OUR  SPONSORS 

BALL  AEROSPACE 


PREMIER  SPONSOR 

Ball  Aerospace  pioneers  discoveries  that 
enable  our  customers  to  perform  beyond 
expectation  and  protect  what  matters 
most.  We  create  innovative  space  solutions,  enable  more 
accurate  weather  forecasts,  drive  insightful  observations  of 
our  planet,  deliver  actionable  data  and  intelligence,  and  ensure 
those  who  defend  our  freedom  go  forward  bravely  and  return 
home  safely. 

Ball  Aerospace  located  near  Wright-Patterson  Air  Force  Base, 
supports  the  missions  of  the  Air  Force  Research  Laboratory 
(AFRL),  the  National  Space  Intelligence  Center  (NASIC),  and 
several  Air  Force  Life  Cycle  Management  Center  (AFLCMC) 
Program  Executive  Officer’s  programs.  Ball  is  a  prime 
contractor  providing  research  and  development  in  partnership 
with  the  Airman  Systems  Directorate  (RH)  and  AFRL  to 
discover,  develop,  and  integrate  affordable  technologies 
to  improve  Warfighter  performance,  exploit  autonomous 
systems  and  enhance  Airman-machine  teaming  in  Air,  Space 
and  Cyberspace.  In  collaboration  with  RH,  Ball  provides  the 
Special  Forces  and  Intelligence  Communities  with  innovative, 


human-centered  solutions  to  complex  customer  challenges 
and  creates  new  warfighting  capabilities.  We  work  with  RH 
and  AFRL  across  multiple  research  programs  to  ensure  that 
future  Airmen  -  through  training  and  technology  -  will  work 
effectively  and  responsively  with  autonomous  teammates  in 
highly-contested,  dynamic  environments  leveraging  integrated, 
multi-domain  operations. 

Ball’s  innovative  technology  can  be  found  anywhere,  from  right 
here  on  Earth  to  millions  of  miles  in  deep  space.  An  industry 
leader  with  proven  quality  and  performance,  Ball  develops 
spacecraft  and  space-based  instruments,  tactical  defense 
products  and  geospatial  information  solutions.  Be  it  space 
exploration,  Earth  and  space  science,  commercial  remote 
sensing  or  national  security  and  intelligence,  we  deliver  end-to- 
end  capabilities  for  mission  success. 

Ball’s  innovation  delivers  science  and  technology  programs, 
engineering,  analysis,  and  operations  to  these  customers  on 
time  and  within  budget.  To  learn  more  about  Ball,  visit  the 
company  website  at  www.ball.com/aerospace 


ROCkWCU  ROCKWELL  COLLINS 

Collins  elite  SPONSOR 

As  a  real-world  original  equipment  manufacturer  and  simulation 
and  training  provider,  Rockwell  Collins  provides  a  true  end- 
to-end  package,  bringing  real  platform  equipment  into  the 
virtual  environment.  We  enable  our  customers  to  securely 
and  effectively  train  across  the  spectrum  in  live,  virtual  and 
blended  environments.  We  provide  training  environments  that 
are  highly  scalable,  immersive,  realistic  and  seamless  across 
live  and  synthetic  training  domains  -  all  contributing  to  the 
optimal  readiness  of  operators  and  support  personnel.  Our 
high  fidelity  integrated  simulation  solutions  help  our  customers 
enhance  mission  effectiveness,  whatever  the  scenario. 
Leveraging  our  strengths  in  avionics  systems  development  and 
training  systems  design,  we  deliver  integrated  solutions  that 
span  the  entire  training  continuum,  from  instructional  systems 
design  through  fully  integrated,  high-fidelity  simulators,  and 
into  the  Live,  Virtual  and  Constructive  scenarios  to  blend  the 
real  world  with  the  virtual  world.  To  learn  more,  please  visit: 
rockwellcollins.com 


Our  capabilities  and  solutions  include: 

•  Training  and  publications  development 

•  Operator  and  maintenance  training  solutions 

•  Virtualized  avionics  and  procedural  training 

•  Developmental  simulation  framework  Helmet  Mounted 
Displays  (HMDs)  and  soldier  display  systems 

•  Visual  system  solutions  including:  image  generation, 
displays,  projection  systems  and  correlated  sensors 

•  Tactical  Air  Combat  Maneuver  Instrumentation  systems 

•  Live  and  virtual,  augmented  and  mixed  reality  training 
applications 
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DCS CORPORATION 

CONTRIBUTING  SPONSOR 


r  DCS  is  an  employee-owned  engineering  firm 
with  40  years  of  experience  serving  DoD  communities.  The 
company’s  1100  employees  provide  innovative  solutions 
throughout  all  stages  of  DoD  technology  maturation  and  life 
cycle  management.  DCS  has  an  extensive  array  of  experience 
and  capability  in  human  systems  integration  and  associated 
research.  This  includes  leadership  of  the  Cognition  and 


Neuroergonomics  Collaborative  Technology  Alliance  (CaN 
CTA),  a  US  Army  funded  international  partnership  of  academic, 
industry  and  government  collaborators  conducting  advanced 
neuroscience  research.  DCS  also  conducts  research  and 
development  of  human  systems  interfaces  for  US  Army,  Air  Force 
and  Navy  including  research  in  human  autonomy  teaming. 


NOTES 
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MARK  YOUR  CALENDAR! 


ARMY  SCIENCE  &  TECHNOLOGY 

SYMPOSIUM  AND  SHOWCASE 

Emerging  technologies  for  the  future  force 

August  21-23,  2018 

Washington  Convention  Center,  Washington  DC 

NDIA.org/Army-Science 


Seeing  into  the  black  box: 

Using  eye  tracking  in  user-driven  workflows  to 
better  understand  decision-making  processes 


Kristin  M.  Divis  (kmdivis@sandia.gov  |  505.284.6561) 

Max  G.  Chen,  Laura  A.  McNamara,  &  J.  Dan  Morrow 


Sandia 

National 

Laboratories 


ENERGY 


*  r  Tf' 


Sandia  National  Laboratories  is  a  multimission  laboratory  managed  and 
operated  by  National  Technology  and  Engineering  Solutions  of  Sandia  LLC,  a 
wholly  owned  subsidiary  of  Honeywell  International  Inc.  for  the  U.S. 
Department  of  Energy’s  National  Nuclear  Security  Administration  under 
contract  DE-NA0003525. 


SAND  18-771612 


Information  ForagingTheory 


•  Human  ‘informavores’ 

•  Valuable  information  isn’t 
distributed  evenly 

•  We  look  for  proximal 
semantic  cues  to  navigate  to 
patches  of  interest 

•  Effective  foraging  strategies 
maximize  the  value  of 
information  gained  relative 
to  effort  finding  the 
information 


(see  Pirolli  &  Card,  1999) 


...so  we  “just”  need  to  capture  markers  of  analysts’ 
search  behavior? 


°  Design  a  simple  signature  detection  task,  invite  SAR  Imagery  Analysts  to  find  the 
targets  (capturing  interaction  logs) 

°  Need  to  know  where  in  an  image  the  analyst  looks  (gaze-contingent  decision 
patterns) 


Great  idea  ...  but  eye  trackers  couldn’t  support  the  workflows  we  wanted  to 
study 


User-driven  dynamism 


The  underlying  problem: 


You’ll  need  a  village 

Static  stimuli  manipulated 

by  user,  all 

spatial/temporal  DOF 

• 

Interactive  SAR 

workflow 

Standard 

Single  stimulus,  serial 
presentation,  minimal 
user  interaction-  visual 
image  inspection 


Don’t  even  go  there 

Dynamic  content 
under  full  user 
control 

GMTI  analysis 


Difficult  but  possible 

Dynamic  content, 
minimal  user  control  over 
display  -  watching 
movie 


Dynamism  intrinsic  to  stimulus 


Display  World: 


Eye  Tracker  World: 


Dynamic,  user- 
driven  content 


(x.y.t) 


Semantics  /  Gaze  X 


sfls  00 

Semantics  X  Gaze  V 


What  do  we  need? 


Ingredients  for  a  Simple  Gaze-Informed  Foraging  Model: 


Dynamically  changing  content - > 

Gaze  tracked  against  display  surface - > 

User  operations  (mouse,  keyboard) - > 

Application/content  log  files - > 


n 


VECTOR  OF  EVENTS 

integrated  into  a  single  timeline  with  minimal  latency 


Step  I :  Data  “Sandbox”  Creation 


Synthetic  Aperture  Radar  (SAR)  images  with  rich 
metadata 

•  Each  pixel  tagged  with  info  such  as  geospatial  location 
Superpixel  and  megapixel  algorithms  established 

4  constrained  visual  search  task  (zoomed  in;  click  and 
drag  to  pan)  to  establish  ground  truth 


Superpixel  Example 
(Moya  et  al.,  2014) 


Step  2:  Sequence  of  Events 


Map  data  streams  into  a  sequence  of 
events. 

Requires: 

•  Integration  of  data  streams 

•  Dimensionality  reduction 

•  Temporal  merging  of  data  streams  into  a 
single  sequence 


Ingredients  for  a  Simple  Gaze-Informed  Foraging  Model: 

Dynamically  changing  content - ► 

Gaze  tracked  against  display  surface - ► 

User  operations  (mouse,  keyboard) - ► 

Application/content  log  files - ► 

VECTOR  OF  EVENTS 

integrated  into  a  single  timeline  with  minimal  latency 

T0  Tn 


Gaze  data:  Integration  and  dimensionality  reduction 


Starting 

place: 


Display  (Pixels):  Eye  Tracker: 


•  (x,  y.  t) 


Raw  eye  movement  data  (x,y,t)  at  every  sample  (60 
Hz)  in  screen  space 

Image  coordinates  of  upper  left  of  display 


Simple 

transformations: 


Display  (Geospatial): 


Eye  &  Geospatial: 


Fixations  in  Geospatial: 


Activated  Superpixel: 


Transform  image  coordinates  to  underlying 
geospatial  coordinates  (encoded  in  each  pixel  of  SAR 
image) 

Transform  raw  eye  movement  data  in  screen  space 
to  eye  movement  data  in  geospatial  coordinates 


Move  from  raw  eye  movement  data  to 
characterized  gaze  patterns  (i.e.,  fixations) 

Move  from  fixations  in  geospatial  coordinates  to 
gaze-contingent  activated  superpixels 


Dimensionality 

Reduction: 


Log  files:  Integration  and  dimensionality  reduction  ® 

Two  additional  data  streams  of  interest:  mouse  (panning  behavior)  and  keyboard*  (“ground  truth”) 


Raw  data: 


Coordinates  of  upper  left  corner  of  screen  as  participant 
pans  (click  +  drag)  to  explore  zoomed  image 


Discretize  into 
runs  &  pauses: 


Pause  (not  moving): 


•  Short  (<  1 500  ms) 

•  Long  (>  1 500  ms) 


Run  (moving): 


•  MagnituderTotal  distance  (3) 

•  Angle:  angle  from  start  to  end  (4) 

•  Length  ratio:  Linearity  or 
“curviness”  (3)* 


...and  now  we  have  two  critical  data  streams  reduced  in  dimensionality  to  a  manageable  space! 


Merge  data  streams  into  single  sequence 


Discretized 

panning  &  gaze 
data: 


Runs  (magnitude,  angle,  length  ratio),  short  pauses,  long  pauses) 


140 


Activated  superpixels 
(fixations) 


Temporal 
alignment  of 

panning  &  gaze 
data: 


Merge  into  single 
sequence  of  tokens  (0- 
255),  replacing  pauses 
with  fixations: 


67-  7  -  53  -  136  -  I  -  42  -  I  -  188-  7-  78-  203  -  91  -  116  -  I  -  26-8 


Step  3:Analyze  sequences 


...  we  now  have  two  critical  data  streams  reduced  in  dimensionality  and 
integrated  into  a  single  sequence  of  events! 

•  Treat  sequences  as  a  vocabulary  of  analyst  interactions  (e.g.,text  analytics,  n-grams) 

•  Identify  decision-making  patterns  and  cues  to  interactions  of  interest  (e.g.,  found  a 
target  or  will  find  a  target  soon) 

Markers  of  “good”  or  “bad”  performance 
o  Characterize  individual  analyst  patterns  of  behavior 

Characterize  behavior  associated  with  a  particular  task,  regardless  of  analyst 


O 


Now  what? 


Refine,  expand,  and  validate/verify  our  process 

•  End-to-end  thinline  prototype 

•  Narrow  in  on  best  features  to  include  (e.g.,  better  ways  to  discretize  the  data  streams?) 

•  Move  all  the  way  to  meaningful,  semantic  content 

o  We’ve  made  it  to  superpixels,  but  what  about  “that’s  a  car”? 
o  Algorithms  developed  at  Sandia  can  help  with  this 

•  Formally  pull  in  ground  truth  data 

•  Validate  and  verify  on  new  data  set 

o  Initial  analyses  focused  on  Task  I  (still  have  all  ofTasks  2-4) 

Collect  additional  data  (same  tasks,  new  tasks,  expert  population,  beyond  SAR,  etc.) 


Questions? 


Kristin  M.  Divis  |  Sandia  National  Labs  -  New  Mexico  |  kmdivis@sandia.gov  |  505.284.6561 


Human  Systems 
Community  of  Interest 
Government  Panel 


Distribution  Statement  A:  Approved  for  Public  Release.  Distribution  Unlimited.  ONR  DCN#:  43-3712-18. 


1 


Human  Systems  Community  of  Interest 

Active  Membership 


STEERING  GROUP 

Dr.  John  Tangney  (Navy) 

Dr.  Bindu  Nair  (OSD) 

Mr.  John  Lockett  (Acting)  (Army) 
Dr.  Kevin  Geiss  (AF) 

Dr.  Michelle  Sams  (Army) 

Mr.  Doug  Tamilio  (Army) 

Dr.  Patrick  Mason  (Navy) 

Ms.  Lisa  Sanders  (SOCOM) 

WORKING  GROUP 


Personalized  Assessment, 
Education,  and  Training 

Dr.  Glenn  Gunzelmann  (AF) 
Mr.  Rodney  Long  (Army) 

Dr.  Kendy  Vierling  (USMC) 

Dr.  Ray  Perez  (Navy) 

CAPT  Sidney  Fooshee  (OSD) 
Dr.  Sae  Schatz  (ADL) 

Dr.  Marty  Bink  (Army) 


Dr.  Todd  Nelson  (AF) 

Dr.  Marty  Bink  (Army) 

Dr.  Paul  Chatelier  (Navy) 
CAPT  Sidney  Fooshee 
Ms.  Karen  Gregorczyk  (Army) 


SUB-AREAS 


Dr.  Kelvin  Oie  (Army) 

LCDR  Pete  Walker  (Navy) 

Ms.  Cheryl  Stewardson  (Army) 

Ms.  Josephine  Wojciechowski  (Army) 
Dr.  Ben  Petro  (OSD) 


Protection,  SustainmenT 
and  Warfighter  Performance 

Dr.  Mike  LaFiandra  (Army) 

Dr.  John  Ramsay  (Army) 

Dr.  Peter  Squire  (Navy) 

Ms.  Stephanie  Miller  (AF) 

Dr.  Lloyd  Tripp  (AF) 

Dr.  John  Schlager  (AF) 

Ms.  Roxanne  Constable  (AF) 
Dr.  Karl  Van  Orden  (Navy) 


Systems  Interfaces  and  Cognitive 

Processes 

Dr.  Todd  Nelson  (AF) 

Dr.  Susan  Hill  (Army) 

Dr.  Micah  Clark  (Navy) 

Dr.  Mark  Derriso  (AF) 

Dr.  Erica  Johnson  (AF) 

Dr.  Caroline  Mahoney  (Army) 

Dr.  Jennifer  Serres  (AF) 

Mr.  Ed  Davis 
Dr.  Tom  McKenna 


SICP  (cont’d) 

Dr.  Liz  Bowman  (Army) 

Dr.  David  Scribner  (Army) 

Dr.  Rebecca  Goolsby  (Navy) 
Mr.  Eric  Hansen  (AF) 

Dr.  Edward  Palazzolo  (Army) 
Dr.  Lisa  Troyer  (Army) 

Dr.  Laurie  Fenstermacher  (AF) 
Dr.  Adam  Russell  (DARPA) 
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Human  Systems  Community  of  Interest 

Vision  and  Goals 


Vision: 

Develop  and  deliver  new  human-centered 
technologies  to  quantify  mission 
effectiveness  and  to  select,  train,  design, 
protect,  and  operate  for  measurably 
improved  mission  effectiveness. 


Goals  -  to  enhance  mission  effectiveness 

•  Integrated  simulations  for  mission  training  and  experimentation 

•  Human-machine  designs  for  mission  effectiveness 

•  Assessment  of  (candidate)  operator  effectiveness 

•  Operating  through  battlespace  stresses 

•  Mastering  the  PMESII*  battle  space 

*Political,  Military,  Economic,  Social, 
Infrastructure,  &  Information 
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State  of  Technology:  Focus  Areas 


Note:  Slide  as  example  only 


Personalized  Assessment,  Education,  and  Training  Protection,  Sustainment,  and  Warfighter  Performance 


Right  Person,  Right  Job,  Right  Skills 

>  First  Principles  for 
Training  Design 

•  Personnel  Selection 
and  Assignment 


Ensuring  Warfighter  Safety  and  Survivability 

•  Understanding  and 
Quantifying  the  Effects 
of  Critical  Stressors 

•  Critical  Stressor 
Mitigation  Strategies 


System  Interfaces  and  Cognitive  Processes 


Effective,  Natural  Human-Machine  Teaming 
Human-Machine  Teaming 
Intelligent,  Adaptive  Aiding 
Human  Information  Interpretation  &  Influence 
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Operational  Concept 

Mission  Effectiveness  Quantification 

Capability:  Integrated,  persistent  Live- Virtual-Constructive  (LVC)  training  environments 
incorporating  adaptive  training  methods  to  accelerate  Service,  Joint,  and  Coalition 
Readiness 


Affordable  Mission  Realism  -  Integrated  Forces  -  Quantified  Effectiveness 
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Human-Machine  Teaming  Collaboration; 

Combat  Teaming 


Human  Systems  COI  S&T  Focus  Areas  that  Address  Human-Machine  Teaming 

1.  Learning  Machines 

■  Computational  Models  of  Human  Cognitive,  Psychomotor,  and  Perceptual  Capabilities 

2.  Human-Machine  Collaboration 

■  Intuitive,  Multi-sensory,  Adaptive  Interfaces 

■  Natural  Language  Interfaces 

3.  Assisted  Human  Operations 

■  Intelligent,  Adaptive  Aiding 

4.  Human-Machine  Combat  Teaming 

■  Trust  Calibration  and  Transparency  of  System  Autonomy 

■  Metrics  of  Mission  Effectiveness  at  Individual  and  Unit  Level 

5.  Autonomous  Weapons 

■  Systems  that  can  take  action,  when  needed 

■  Architectures  for  Autonomous  Agents  and  Synthetic  Teammates 


. . .  and  Experiments  Using  Realistic  Mission  Scenarios 
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Service  Demand  Signals 


Personalized  Assessment, 
Education  and  Training 


Protection,  Sustainment  and 
Warfighter  Performance 


❖  Personalized,  integrated  assessments  and 
training  to  improve  performance,  accelerate 
proficiency  and  increase  affordability 

❖  Enhanced  warfighter  performance 
through  scenario  based  training  & 
automated  performance  based 
readiness  assessments 

❖  Maintain  air  superiority  over  complex, 
evolving  threats  using  adaptive  training 


❖  Greater  force  protection  to  ensure 
survivability  across  all  operations  and 
environments 

❖  Maintain  health  &  injury  recovery; 
reduce  noise  induced  hearing  loss 

❖  Agile  Combat  Support  through 
countering  aerospace  physiology 
and  toxicology  threats,  reducing 
cognitive  workload 


System  Interfaces  and  Cognitive  Processing 


❖ 


❖ 


Achieve  operational  maneuverability  through  soldier-system  integration 

Design  systems  to  enable  effective  human  machine  interaction,  including  robotics  & 

autonomous  systems 

Enhanced  interaction  &  trust  w/  autonomous  systems;  increased  SA  for  operators; 
reduced  analyst  workload 

Provide  situational  awareness;  timely  mission  command  and  tactical  intelligence  human 
agent  teaming 

Army  Enduring  Challenges 
Navy  Vision/Objectives 

AF  Core  Mission/Challenges 
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COI-to-COI  Collaborations 


•  ASBREM 

•  Human  Performance  Optimization  Committee 

•  Joint  Biomedical  Modeling  and  Simulation  Initiative 

•  Walter  Reed  Army  Institute  of  Research  (WRAIR)  evaluating  TAPAS  as  a  contributor 
toward  predictors  of  mental  health  &  medical  attrition 

•  ASBREM,  Sensors,  CWMD 

•  Wearable  Physiological  Monitors 

•  Autonomy 

•  Roadmap  development:  Human-Machine  Teaming  shared  area 

•  V&V  Licensing  Study 

•  Executing  Joint-Service  Autonomy  Research  Pilot  Initiatives 

•  Cyber 

•  Cyber  Selection  and  Training 

•  Cyber  Situational  Awareness 

•  CWMD 

•  Dark  web  concerns,  social  network  analysis,  and  counter-terrorism  research 
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COI  Activity  2017 


Major  Accomplishments 

•  IMPACT  DoD  Virtual  Lab  allowed  one  operator  controls  1 2  vehicles  -  force  multiplier,  lauded  by  OSD  as 
Autonomy  Re-search  Pilot  Initiative  (ARPI)  poster  child 

•  Advanced  technologies  for  Battlefield  Airman  (BATMAN)  resulted  in  30  tech  transitions  to  Special  Tactics 
operators  in  advanced  audio/visual  communications  and  lightweight  equipment,  mission  rehearsal,  and  multi¬ 
patient  monitoring.  Transitioning  Battlefield  Airmen  Trauma  Distributed  Observation  Kit  capability  to  Army  and 
joint  medical  community. 

•  Augmented  Immersive  Team  Training  (AITT)  provided  a  unique  JTAC  training  capability  for  Joint  Staff  exercises. 
Scheduled  to  transition  to  multiple  Marine  Corps  programs. 

•  Multiple  Live  Virtual  Constructive  (LVC)  product  transitions;  Deployable  LVC  baseline,  Learning  Management 
SOA  Advanced  Technology  Demonstration  completed 

•  Generalized  Intelligent  Framework  for  Training  transitioned  to  900  users  in  53  countries 

•  DoD  Wearables  Technology  Workshop  Feb  2017 

Engagements  with  organizations,  individuals,  entities  outside  DoD 

•  2017  NDIA  HS  Conference  well  attended 

•  2017  IR&D  event  20-23  June  ,  3rd  Biennial  (27  Companies  participated) 

•  HS  COI  sub  area  meetings  at  l/ITSEC 

•  Digital  and  Social  Media  Assessment  course  at  National  Defense  University  with  NATO  attendees 

•  Ongoing  Metrics  White  paper  discussions  Between  PAET  &  Industry 

•  2017  All  Hands  COI  meeting,  NASA,  DOE,  DHS  presented 

•  Aerospace  Medical  Association  Annual  Meeting  April  2017 
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Impact  of  Human  Systems 
Community  of  Interest 
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SUB-AREA  S&T  THRUSTS 
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Personalized  Assessment, 
Education,  and  Training 
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PAET  Scope 

Personalized  Assessment,  Education  & 

Training  (PAE&T) 


Research  and  development  in  personnel  assessment  will  produce  integrated  measures  and 
adaptive  testing  for  more  precise  assessment  of  individual  potential,  yielding  improved 
personnel  selection  and  assignment.  Meanwhile,  work  in  education  and  training  will  produce 
competency-based  systems  grounded  in  quantitative  metrics  to  enable  personalized, 
proficiency-based  training  to  accelerate  acquisition  and  enhance  operational  performance. 
The  end  result  is  more  capable  warfighters  with  decreased  training  costs. 


Thrust  Area  1 : 

Training,  Education,  and  Personnel 

Development 

S&T  Focus  Areas  on  Roadmap: 

•  Realistic,  secure,  and  adaptive  LVC 
environments 

•  Persistent  and  personalized  readiness 
assessment  and  tracking 

•  Multi-Level  modeling  for  readiness 
management 

•  Computational  cognitive  science  research 
to  support  model  and  agent  development 
for  training  and  operational  support 


Thrust  Area  2: 

Personnel  Selection  and  Assignment 

S&T  Focus  Areas  on  Roadmap: 

•  Predictors:  Expand/refine  non-cognitive 
measures  (e.g.,  Tailored  Adaptive 
Personality  Assessment  System) 

•  Outcomes:  Expand/refine  behavior  and 
performance  data 

•  Models:  Expand/refine  predictive  analytic 
model  for  integrated  cognitive  plus  non- 
cognitive  measures  to  predict  attrition, 
performance,  and  behaviors 
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HUMAN  SYSTEMS  COI  SUB-AREA: 

Personalized  Assessment,  Education,  and 

Training 

VISION 

A  readiness  ecosystem  that  ensures  the  right  person  has  the  knowledge , 
skills,  and  experiences  needed  to  be  mission  ready  for  a  dynamic  and 
uncertain  21st  century  operating  environment 
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Our  Story 


•  Challenges 

•  More  unpredictable  and  asymmetric 
adversaries 

•  Dynamically  evolving  operational 
environments 

•  One  solution  does  not  meet  all 

•  Reduced  manning 

•  Diverse  personnel  pool 

•  Budget  &  manpower  constraints 

•  Better  training  at  point  of  need 


•  S&T  Solutions 

•  Adaptive  LVC  environments  that 
keep  pace  with  operations 

•  Human  Science  models  for 
assessment  and  training 

•  Performance  measurement  and 
assessment  to  tailor  training 

•  Cognitively-based  instruction, 
agents  and  training  schedules 

•  Science  of  Learning 


} 


More 

capable 

Warfighters 


Today 


Future 


Improved  ■ 
Selection  & 
Assignment 


Combat  Mission  Ready 


Increased 

Apprenticeship 

i 


Time 


Distribution  Statement  A:  Approved  for  Public  Release.  Distribution  Unlimited.  ONR  DCN#:  43-3712-18. 


15 


Thrust  1:  Training,  Education,  and 
Personnel  Development 


Delivering  the  Mission 


Delivering  Capability  (i.e.,  End  States) 


Education  &  Training  Practices  and  Technologies 
that  Support  Efficient  and  Effective  Development 
of  Mission  Readiness  and  Cognitive  Agility 

•  Leverage  learning  sciences  and  technology  to 
reduce  resource  costs  (cost,  manpower,  time) 

•  Tailor  training  to  individuals  to  enhance  warfighter 
capabilities  and  agility 

•  Measure,  track,  &  warehouse  quantitative, 
proficiency-based  performance  measures 

Key  Technical  Challenges 


•  Persistent,  interoperable  learning  “ecosystem” 

...with  personalized  measurement:  readiness  tracking 

•  Secure  LVC  ioint/coalition  training  environments 

. .  .with  realistic  constructive  teammates  /  adversaries 

•  Consistently  high-quality  training  and  education, 
tailored  to  individuals  and  available  when  needed 

•  Increased  insight  into  personnel  (data)  informs 
individual  learning  decisions  and  mission  planning 


Example  Program  Successes 


•  Developing,  deploying,  and  using  proficiency-based 
performance  measures  /  analyses 

•  Warehousing  &  using  (big)  learning  data  to  inform 
life-long  learning  and  operational  decisions 

•  Securely  integrating  LVC  environments 

•  Develop  adaptive  and  valid  cognitive  agents 

•  Adapting  learning  sciences  to  military  contexts  and 
foster  the  right  culture  for  their  use 


Txn 


AITT 
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Training,  Education,  and  Personnel 

Development 

Roadmap 


Mission  Education  &  Training  Practices  and  Technologies  that  Support  Efficient  and  Effective 
Need  Development  of  Mission  Readiness  and  Cognitive  Agility 


Military  Secure  LVC  joint/coalition  Individually  tailored  training  Persistent,  interoperable 

Capabilities  training  environments  and  education  learning  “ecosystem” 


2016  2018  2020  2022  2024  2026 
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Training,  Education,  and  Personnel 

Development 

Program  Detail  (1) 


S&T  Focus  Areas 

Near-term 

Mid  /  Far-term 

Operational 

Opportunities 

FY 16  FY 17  FY 18  FY 19  FY  20 

Secure  Environments  for  LVC 

Traininq 

Adaptive  LVC  Training  for  Enhanced  Warfighter  Readiness  V* 

Seamless  integration  of  live,  virtual, 

&  constructive  training 
environments;  personalized  training 
grounded  in  operationally  relevant 
proficiency  assessments;  Range 
infrastructure  to  support  LVC 
integration  for  4th/5th  gen  aircraft; 
air-ground  simulation  integration, 
scalable,  adaptive  constructive 
agents  that  think  and  act  like  people 
to  support  training  &  ops 

Develop,  validate,  and  demonstrate 
seamlessly  integrated  Live,  Virtual, 
and  Constructive  (LVC) 
components  into  persistent,  secure, 
and  manageable  training  and 
operations  environments  across  the 
Range  Of  Military  Operations 
(ROMO) 

Adaptive  Training  for  C4ISR 

Secure  LVC  Advance  Traininq  Environment. 

EDUCAT2E  (see  notes)  © 

Live  Virtual  Constructive  Simulation  & T raininq  H 

Live,  Virtual,  Constructive  Training  Fidelity 

Future  Integrated  Traininq  Environment  © 

Infrastructure  for  Performance 

Measurement,  Trackinq,  and 

Autonomous  Models  and  Agents  for  Training  &  Operations  W 

Formal,  quantitative  measures  of 
proficiency;  Embedded  performance 
measures  in  training  environments 
and  operational  systems;  Integrated 
and  persistent  warehousing, 
diagnosis,  and  assessment  of 
readiness  to  drive  personalized 
training  requirements  and  career- 
long  readiness  management 

Personalized  Traininq 

Quantitative,  embedded 
performance  measures  in  training 
and  operational  systems,  combined 
with  warehousing  capabilities  and 
metrics  to  assess  mission  readiness 
and  effectiveness  over  a  career 

Learning  Continuum  and  Performance  Aid  & 

Adaptive  LVC  Traininq  for  Enhanced  Warfiqhter  Readiness  W 

Adaptive  Traininq  Research  E 

Total  Learnino  Architecture 

Complex  Coqnitive  Skills  E3 
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Training,  Education,  and  Personnel 

Development 

Program  Detail  (2) 


S&T  Focus  Areas 

Near-term 

Mid  /  Far-term 

Operational  Opportunities 

FY 16  FY 17  FY 18  FY 19  FY  20 

Adaptive,  Multi-level  Constructive 

Models 

More  robust,  valid,  &  Integrated 
mechanisms  thatenable 
constructive  agents  that  truly  think 
and  act  like  people  for  training  and 
operational  applications; 

Incorporate  robust  capabilities  for 
situation  representation  and 
language  processing  to  support 
human-machine  teaming. 

Adaptive  LVC  Training  for  Enhanced  Warfighter  Readiness''*/1 

Realize  the  full  potential  of  LVC  to  provide 
personalized,  anytime,  anywhere  training. 
Reduce  manpower  costs  for  training  with 
increased  adaptivity  in  constructive  forces. 
Enhanced  validity;  increased  cognitive  & 
behavioral  fidelity;  agents  that  are 
language  enabled  &  situationally  aware; 
consideration  of  cognitive  moderators; 
emphasis  on  complex  &  uncertain 
operational  contexts;  also  facilitates 
human-machine  teaming 

Autonomous  Models  and  Agents  forTraining  &  Operations  t  j 

Computational/Cognitive  Models  for  ITS  © 

Live,  Virtual,  Constructive  Training  Fidelity  £5) 

Human-Machine  Training  & 

Assessment 

We  must  prepare  for  the  need  to 
train  autonomous  systems  for  the 
same  operational  realities  as 
humans,  including  training  with 
human  teammates;  requires 
bridging  among  machine  learning, 
large  scale  computing,  and  LVC. 

Autonomous  Models  and  Agents  forTraining  &  Operations 

Training  to  tailor  baseline  autonomous 
systems  for  specific  capabilities, 
environments,  and  operations.  Integration 
of  large-scale  computing  resources  with 
machine  learning  capabilities  in  LVC 
training  environments  will  be  a  game 
changer  in  human-machine  teaming  by 
providing  the  opportunity  for  humans  and 
machines  to  train  separately  and  together 
in  the  same  environments  to  prepare  for 
the  uncertainty  of  real  operations. 

Adaptive  LVC  Training  for  Enhanced  Warfighter  Readiness  W 
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Training,  Education,  and  Personnel 

Development 

Program  Detail  (3) 


S&T  Focus  Areas 

Near-term 

Mid  /  Far-term 

Operational 

Opportunities 

FY 16  FY 17  FY 18  FY 19  FY  20 

Learning  Sciences  for  Military 

Training 

Adapt  research  on  learning  and 
training  to  the  unique  requirements 
of  the  military  environment 

Adaptive  LVC  Training  for  Enhanced  Warfighter  Readiness  W 

A  significant  scientific  base  exists  in 
the  learning  sciences,  but  most  of  it 
has  been  developed  in  non-military 
contexts.  Adapting  and  extending 
existing  research  for  the  unique 
requirements  of  military  training  will 
improve  its  efficiency  and 
effectiveness 

Adaptive  Trainina  Research  P 

Total  Learninq  Architecture 

Personal  Assistant  for  Learninq 

Accelerating  the  Development  of  Small  Unit  Decision-Makers 
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Technology  Highlight:  Predictive  Performance 

Optimization  (PPO) 


Program 

•  Use  quantitative  models  of  human 
learning  and  forgetting  to  optimize  and 
personalize  training  schedules 

Status  &  Impact 

•  Improves  training  efficiency 

-  Shifts  from  calendar-based 
training  to  cognitively  principled 
personalization  schedules 

-  Minimizes  training  costs/time 
while  maximizing  performance 
effectiveness 

•  More  effective  acquisition  of  training 
objectives  &  more  sustained  proficiency 


•  Demonstrated  to  maximize  performance 
effectiveness  while  simultaneously 
reducing  time/costs  in  training 
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What?  Developed  by  the  Advanced  Distributed 
Learning  (ADL)  Initiative,  Experience  API  (xAPI)  is 
a  technical  specification  that  facilitates  the 
standardized  documentation  and  interoperable 
communication  of  learning  experiences  (i.e. 
data)  among  disparate  software  systems. 
Essentially,  it  helps  breakdown  data  stovepil 
between  education  and  training  technology 

Why?  xAPI  helps  fuel  learning  analytics,  not 
within  a  single  training  system  or  course,  but 
potentially  across  someone's  entire  lifelong 
learning  set  of  experiences.  Today,  xAPI  has  been 
integrated  into  numerous  COTS  systems  and 
demonstrated  in  various  DoD/Federal  settings. 
Even 

about  how  xAPI-based  data  will  soon  replace  the 
standard  resume! 


O  GitHub  -  adlnet/xA  x 

C  A  GitHub,  Inc.  [US]  https://github.com/adlnet/xAPI-Spec 


o 


Personal  Open  source  Business  Explore 


Pricing  Blog  Support  This  re 


2  adlnet/  xAPI-Spec 


H  Pull  requests  0 


l)  Projects  0 


[ih  Graphs 


No  description  or  website  provided. 


©  1,803  commits 


Branch:  master  » 


p  7  branches 


il  32  contributors 


andyjohnson  committed  on  GitHub  Update  CONTRIBUTING.md  ... 
|  logos 


Add  various  files  from  master 


0  .gitignore 
0  CONTRIBUTING.md 
0  README.md 
0  xAPI -About,  md 
0  xAPI-Communication.md 
0  xAPI-Data.md 
0  xAPl.md 
0  xAPIDataFlow.jpg 
0  xAPIMetadata.jpg 
0  xAPIPersonasjpg 


Added  prototype  auto-tagging  script 
Update  CONTRIBUTING.md 
resolved  conflicts  in  readme  and  contributing 
Update  xAPI-About.md 

Merge  pull  request  #986  from  garemoko/issue-959 
Merge  pull  request  #985  from  garemoko/issue-982 
remove  “(xAPI)”  in  part  2  header 
updated  dataflow  and  metadata  images 
updated  dataflow  and  metadata  images 
updated  graphics 


Latest  commit  H55c3b  13  days  ago 
a  year  ago 
3  months  ago 
13  days  ago 
1 3  days  ago 
13  days  ago 
27  days  ago 
27  days  ago 
a  month  ago 
3  months  ago 
3  months  ago 
3  months  ago 


Who  Cares?  Using  xAPI  will  allow  multiple, 
disparate  learning  devices  (e.g.,  e-learning,  mobile 
learning,  simulations,  physical  sensors)  to  be  used 
longitudinally  as  a  cohesive  system.  It  enables 
broad  human  performance  data  management  and 
interoperable  exchange.  Ultimately,  this  will  enable 
much  improved  analysis  of  learning/performance, 
better  informing  lifelong  instructional  adaptation 
and  planning,  as  well  as  talent  management 
activities. 


m  README.md 


xAPI-Spec 

This  Github  repository  contains  the  xAPI  Specification.  xAPI  is  a  learning  technologies  interoperability  specification  that 
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Technology  Highlight:  Secure  LVC  Advanced 

Training  Environment 
(SLATE) 


Live,  Virtual,  Constructive  Operational  Training 
Advanced  Technology  Demonstration  Funded 

•  $47M  demonstration  of  4th  and  5th  generation  LVC  training 

-Aircraft  software  modifications 

-  Waveform,  Radio 

-  Model  and  data  processor 

-  Multiple  Independent  Levels  Encryption  (MILS) 

-  P5  Pod  and  F35  LRU  form  factor 

-  Enhanced  range  infrastructure 

-  Standards,  data  specifications,  interface  control  docs 

•  Mission  impact  and  effectiveness  use  cases 
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Success  Story:  Computer  Generated  Forces 
Training  Executive  Agent  (TXA) 


Operational  Challenge 


“An  integrated  LVC  training  environment  with 
today’s  battle  complexity  is  essential  to  improving 
proficiency  across  all  current  and  future  mission 
sets.”  (Naval  Aviation  Vision  2016-2025) 


Problem:  High  manpower  to  run  complex  virtual  training 


Objective:  Make  Computer  Generated  Forces  (CGF)  more 
intelligent  and  adaptive  to  training  objectives 

Outcome:  Transitioned  the  Training  Executive  Agent  (TXA) 
into  the  Navy’s  Next  Generation  Threat  System 


S&T  Accomplishments 


■  TXA  monitors  a  training  exercise  and  issues 
“directives”  to  other  CGFs  to  modify  behaviors 
according  to  a  higher  level  scenario  director 
(training  objectives) 

■  TXA  used  in  NIFC-CA  training  scenarios 


■  Exploring  TXA  usage  on  aviation  pods,  thus 
providing  unique  flexible  embedded  training 
capabilities 


Return  on  Investment 


Affordability 

■  Aid  instructors  and  “pucksteers”  who 
dynamically  controls  CGF  during  execution  of 
a  training  scenario. 

■  Reduce  number  of  required  “pucksteers”, 
reduce  overall  training  costs 

Readiness 


■  Provide  trainees  with  tactically  realistic  entities, 
in  realistic  complex  battle  scenarios 

■  Allow  instructors  to  focus  on  trainee,  not  on 
playing  roll  in  scenario 
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©Thrust  2:  Personnel  Selection  and  JF% 

Assignment  Iff# 


Delivering  the  Mission 

•  Initial  Military  Training  attrition  is  -10%  ($1 .7B  cost/yr) 

•  IMT  attrition  could  be  reduced  to  -  8%  (saving  ~.34B/yr) 
if  current  S&T  product  (TAPAS)  was  implemented  to 
assess  personality.  IMT  attrition  could  be  reduced  to 
6%  (saving  $.68B/yr)  with  FY22  S&T  products. 

•  Reduce  negative  behaviors  for  enlisted  by  -5%. 

•  Increase  satisfaction,  performance,  and  retention  in 
critical  specialties  by  -15%. 

Key  Technical  Challenges 

•  Predictor  measures:  Existing  measures  lack 
individualized  precision  and  are  not  integrated. 

•  Outcome  measures:  Performance  and 
behaviors  are  difficult  to  measure  and 
systematically  obtain  over  a  career. 

•  Predictive  models:  Existing  models  are  stove- 
piped  and  based  on  group  probabilities. 


Delivering  Capability 

Maintain  our  competitive  edge  in  Human  Capital 
(Force  of  Future). 

•  Reduce  attrition  and  negative  behaviors  with 
more  precise  assessments  of  candidates  for 
initial  entry  &  job  assignment. 

•  Improve  performance  and  retention  with  an 
emphasis  on  critical  specialties  (e.g.,  cyber) 
through  advancements  in  talent  assessment 


Example  Program  Success 

•  Enlisted  Personnel  Selection  -  TAPAS 
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Personnel  Selection  and 
Assignment 
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Maintain  our  competitive  edge  in  Human  Capital  (“Force  of  the  Future”,  SecDef  18  Nov  2015). 


Reduce  attrition  and  negative  behaviors  in  the  enlisted  Force  with  more 
precise  §ss^ssment  of  candidates  for  initial  entry  &  job  assignment. 

Improve  performance  and  retention  in  critical 
specialties  through  advancements  in  talent 
assessment* 


Piled  ic 


ors: 


Adaptive 


Increase 

i 

Outcomes:  Imp 
Models:  In 


Predictors:  Expand/refine  non-cognitivs  measures  (e.g. 


Personality  Assessment 


Decision  and  integrate  measures. 


ove  me  asurement  of  performan 

A 

egrate  models  for  individual  pro 


Outcomes:  Expand/refine  behavior  &  performance  data 


System) 


ce  &  behaviors. 

i 

babilities. 


Tailored 

•  • 


Models:  Expand/refine  predictive  analytic  model  for  integrated  cognitive  + 
non-cognitive  measures  to  predict  attrition,  performance,  &  behaviors. 


T" 

2016 


— r 

2019 


"T" 

2021 


"T" 

2022 


2025 


Improve  performance  and 
behaviors  with  optimal 
talent  management 
across  a  career 

f 


Compensatory  models 
integrating  all 
predictors  for  wide 
range  of  outcome  data 


Shading  Legend 

Dark:  Funded 

Light:  Partially  funded 
White:  Not  funded 


Distribution  Statement  A:  Approved  for  Public  Release.  Distribution  Unlimited.  ONR  DCN#:  43-3712-18. 


Participation  Legend: 

0  Army 
0  Navy 
Air  Force 
0  Marines 
OSD/Joint 


Personnel  Selection  and  Assignment 

Program  Detail 


S&T  Focus  Areas 

Near-term 

Mid  /  Far-term 

Operational 

Opportunities 

FY  15  FY  16  FY  17  FY  18  FY  19 

Predictors 

Expand  and  refine  non- 
cognitive  measures 
(temperament,  interests)  and 
specialized  cognitive 
assessments. 

Expand  and  increase  precision  of  Tailored  Adaptive  Personality  AssessmentE 
Develop,  refine,  and  validate  Vocational  Interest  Inventories  0^j  ,-f 

Develop  and  refine  specialized  selection  tests  (e.g..  Cyber,  UAS)  00y 

Personnel  Measures  Research  0 

rcr^Tl 

More  precisely  and  fully 
assess  individual  potential 
and  risk. 

Selection  for  UAS  Personnel  (SUPer)  (L) 

Outcomes 

Integrate  the  behavioral  and 
competency  data  that  define 
criterion  job  performance. 

Leverage  Training  S&T  competency  assessments  in  realistic 

Develop,  refine,  and  validate  behavioral  outcome  measures  0 

Readiness  and  Resilience  0 

(qf 'Til 

More  accurately  assess 
performance  and  behaviors. 

Models 

Expand  and  refine  predictive 
analytic  models  for  integrated 
personnel  measures  to 
predict  attrition,  performance, 

&  behaviors. 

_ 

Predictive  analytical  models  based  on  predictors  and  longitudinal 

Gdi 

Human  Science  Models  0  W 

With  enhanced  Talent 
Management,  improve 
performance,  reduce  attrition 
and  negative  behaviors. 

Success  Story:  Enlisted  Personnel 

Selection 

Tailored  Adaptive  Personality  Assessment  System 


Operational  Challenge 


Increase  precision  of  assessing  individual 
potential,  risk,  and  fit  to  a  military  career. 


•  26  personality  dimensions  such  as  optimism, 
excitement  seeking,  and  non-delinquency 


Applicant  chooses  from  statement  pairs 
generated  on-the-fly  based  on  responses 


S&T  Accomplishments 

State  of  the  art  personality  assessment 
Developed  in  partnership  with  industry 
2009:  Limited  operational  screening  (Army) 
201 0-2011 :  Administered  to  recruits  (Navy) 
2014:  Began  selection  for  5  specialties  (AF) 
2015:  Administered  to  recruits  (Marines) 


Return  on  Investment* 


TAPAS 

Which  of  these  statements 

is  most  like  you? 

•  I  am  not  one  to 
volunteer  to  be  group 
leader,  but  would  serve 
if  asked. 

•  My  life  has  had  about 
an  equal  share  of  ups 
and  downs. 

(example  statement  pair) 


Readiness 

•  Reduces  attrition  by  5% 

•  Reduces  Initial  Military  Training  re-starts  by  3% 

•  Reduces  conduct  incidents  by  5% 


Affordability 

(attrition  cost  -  recruiting,  training) 

•  Current  implementation  saves  ~  $30M/year 

•  Expanded  use  can  save  ~  $50M/year 


*  Based  on  Army  data  for  limited  operational  screening. 
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System  Interfaces  and 
Cognitive  Processes 
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HUMAN  SYSTEMS  COI  SUB-AREA: 

System  Interfaces  &  Cognitive  Processes 


VISION 

Warfighters  teamed  with  machines  through  intuitive,  personalized 
interfaces  that  enhance  warfighters'  mission  effectiveness. 


Operators 


Dismounted 

Operators 
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Thrust  1:  Human-Machine  Teaming 


Delivering  the  Mission 

•  Increased  capability  with  smaller  force  structure  across 
air,  land,  sea,  space,  and  cyber 

•  1  MQ-9  Operator  controlling  7  simulated  MQ-9s 

•  Reduced  ISR  PED  Cell  Operators  from  5  to  3 

•  Closed  Loop  Medical  Technology  Research 

•  USTRANSCOM  Global  Mission  Scheduling  System 

•  Reduced  logistics  and  personnel  footprint ;  reduced 
planned  flying  hours  >2%  saving  $37M/yr 

•  Trusted  synthetic  teammates  that  provide 
recommendations  for  battlespace  operations 

•  Reduced  manpower  and  training  requirements 

•  Ability  to  operate  safely  in  highly  contested  environments 

•  Reduced  exposure  to  personnel 


Key  Technical  Challenges 


Delivering  Capability 

Seamless  human-machine  interfaces  enabling  optimized 
weapon  system  and  warfighter  performance  in  all 
contested  domains  and  mission  environments: 

•  Demonstrate  highly  effective,  agile  human-machine 
teaming 

•  Create  actively  coordinated  teams  of  multiple  machines 

•  Ensure  safe  and  effective  systems  in  uncertain  and 
dynamic  environments 


Program  Overview 


•  Immature  intuitive,  multisensory,  adaptive  interfaces 

•  Lack  of  robust  and  reliable  natural  language 
interfaces 

•  Absence  of  effective  gesture  control  interfaces 

•  Fragile  cognitive  models  and  architectures  for 
autonomous  agents  and  synthetic  teammates 

•  Insufficient  degree  of  trust  calibration  and 
transparency  of  system  autonomy 

•  Immature  decision  support  tools 


•  Human-Robot  Interaction 

•  Multisensory  Perception  and  Data  Presentation  Interfaces 

•  Supervisory  Control  Technology  Integration  and 
Demonstration 
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Human-Machine  Teaming 


Mission 

Need 

Military 

Capabilities 


Actively  coordinated  teams  of  multiple  machines  in 

CONCERT  WITH  HUMAN  TEAMMATES  EXECUTING  DESIRED 
MISSION  EFFECTS 

Human-Robot  Teaming 

Gesture/Limited  Dialogue-enabled  UGV  for 
A  Logistics  Connector  Missions 

a 

CC^BNITIVELY  BAjJfD  VISUALIZATION  TECHNIQUES 


Safe  &  effective  Human-Machine  systems  successfully 

OPERATING  IN  COMPLEX,  DYNAMIC  &  CONTESTED  ENVIRONMENTS 

A  Robotic  maintenance  assistants 


Multi-us 


AU  TONOMOUS 


Se 


/^UTONO^OUS  UxV 


Technical 

Goals 


Naui 


RaL 


S&T 

Focus 


201! 


VlSUALIZATIOI 


ombined  Speech  / 
(Jesture  Interfaces 

A 


User-Sys' 

*  Jk* 


nIeufj  p 


CTIC 


COGNITI 


A 

e  m  Inter/ 

A 

Hul/lhN-RDBOt  I  NT 


A" 


Sy  stem  Trust  &  Transparency 


N  AND  DECISION!  SUPPORT  FO^GILE  C>  AND  CYBER 


GESTURE/NOg-VE 

Trust  C a 


ERACTION 

/w  Architectures  and  iNTfeoF  ate:[ 
istell gent  Systems  ° 

•• 

^/Iission-4pec  me  N  vtuf al  Language  Di]uc 


Human 


Schema  for  DynaMic  Management  of 
us  AiJjto  jomous  Systems' 


HETEROGErflEC 

!a 


ns:  REACTivfe 


IRBAL  INTERACTION 


LIBRaTION 


Multisenscry  Perception  and  Interf/ces 

Fl  sion  Exploitation  Tool  Su  te 
Interface^  to  C2  Information  Syste 


IhTEf 


aA 


Rf[aC^A 


GUE 


R/PLATFQ^M,  SYNCHRONIZED 


\MLESS  TEAMING  OF 


WITH  MANNED 


SYSTEMS 

Machine  Teaming  Interfaces 


A 

I 


A 


i 


OPERATIONS 


Hybrid  Force  Demonstration: 
Combined  Air-Sea-Land 
Engagement 


Natural  LJanguageMQialogue 


»• 


Natural  User-System  Interactions:  Trustworthy  Proactive  interfaces 

. i .  . t . 


SociOrCoGNiTivE  Architectures 


Unrestricted  Natural  Language  Dialogue 


Shading  Legend 

Dark:  Funded 

Light:  Not  funded 

Striped:  Partially 
Funded 


Mission  Pl* 

Inning  and  Scheduling  Toe 

)LS 

|  Closed  Loop  Medical  Technology  Re! 

Search 

ISTRIBUTED  INTELLIGENT  INTERFACES  FOR 

Human-Centric  Info  Systems 


Participation  Legend 
%  Army 
0  Navy 
Air  Force 


2qLl 
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Human-Machine  Teaming 

Program  Detail 


S&T  Focus  Area 

Near-term 

Mid  /  Far-term 

Operational  Opportunities 

FY  15  FY  16  FY  17  FY  18  FY  19 

Mission  Planning  and  Scheduling 

Visual  nteractive  Exploratory  Data  Analysis  W 

Soldier-Centered  Design  Tools  g 

Mission  planning  and  scheduling 
tools  that  simplify  COA  generation 
and  enhance  mission  efficiency. 

Tools 

rcr~.il 

Mission  Planning  and  Scheduling  Tools  ^ 

© 

Interfaces  to  C2  Information  Systems 

Supervisory  Control  Technology  Integration  and  Demonstration  W 

Soldier-Centered  Design  Tools  £3 

Operator-centered  interfaces  to  C 2 
Information  Systems  that 
enhance/multiply  mission 
effectiveness. 

Interfaces  to  C2  Information  Systems  ^ 

Multisensorv  Perception  and 

Interfaces 

Multisensory  Perception  and  Data  Presentation  Interfaces  © 

Human-Robot  Interaction  O 

ter -ti 

Advanced  Technologies  for  Battlefield  Airmen  w 

Novel  multi-modal  human-system 
interfaces  that  enhance  operator 
performance. 

Neurocognitive  Architectures  and 

Integrated  Intelligent  Systems 

ONR  Computational  Neuroscience 

Neurocognitive  architectures  that 
maximize  human-machine  team 
performance. 

Brain-Computer  Interaction  0 

rern 

Human  Insightand  Trust  ^ 

Human-Robot  Interaction 

Human-Robot  Interaction 

Novel-Human-Intelligent  Agent  Interactions  0 

Human  Interaction  with  Adaptive  Automation  .  ^ 

Human-machine  teams  that  can 
successfully  operate  in  an  agile 
fashion  in  an  operational 
environment. 

Closed  Loop  Medical  Technology 

Research 

Closed-Loop  Oxygen  Generation  and  Delivery 

Maximize  patient  care  through 
autonomous  technologies  in 
operational  environments 
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Thrust  2:  Intelligent,  Adaptive  Aiding 


Delivering  the  Mission 

•  Maintain  mission  effectiveness  despite  fluctuating 
demands:  No  mission  degradation  in  a  high  tempo 
environment 

•  Optimized  human-machine  teaming:  Dynamic  workload 
allocation  to  improve  mission  efficiency 

•  Provides  shared  situation  awareness  and  transparency 
between  the  operator  and  the  weapon  system  platform: 
Appropriate  level  of  operator  trust 

•  Optimized  warfighter  readiness  and  enhanced  training: 
Identification  of  relevant  biomarkers  indicative  of  operator 
cognitive  and  physiological  state 

Key  Technical  Challenges 


Delivering  Capability 

Enhance  warfighter  effectiveness  by  coupling 
humans  and  machines  through  the  use  of  intelligent 
adaptive  aids  to  protect  from  being  overwhelmed  by 
complexity  and  workload. 

•  Develop  models  of  perception  and  cognition 

•  Assess  the  functional  state  of  the  operator 

•  Real-time  measurement  and  assessment  of 
warfighter  performance 


Program  Overview 


•  Immature  tools  for  individual  and  team  functional 
state  assessment 

•  Fragile  cognitive  models 

•  Operationalize  minimally  invasive  sensor  suites 

•  To  Identify  the  appropriate  biomarkers  for 
determining  operator  performance 

•  Absence  of  effective  gesture/non-verbal  interfaces 


•  Applied  Computational  Neuroscience 

•  Cognitive  Performance  Optimization 

•  Monitoring,  Predicting,  and  Optimizing  Battlespace 
Awareness 
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Intelligent,  Adaptive  Aiding 


Mission 

Need 

Military 

Capabilities 


Enhanced  Warfighter  Effectiveness  By  Using  Adaptive 
Situational  Aids  and  Tools  For  Mission  Success 


Coupling  of  Real-time,  closed  loop  quantification  of 

THE  WARFIGHTER  AND  MACHINE  TO  ACHIEVE  UNPRECEDENTED 
MISSION  SUCCESS 


Warfighter  State  Assessment  /  Prediction 


t 


A 

TA 


Technical 

Goals 


1 


t 


>k  and  Behavior-Driven 
Assessment  Systems 

*  1 

Minimally  Invasivi 

Sensor  Suites 

* 

^NTIFICATION  Of  BlOMARKfeRS  FOR  COGNITIVE  & 
PHYSIOLOGICAL  STAT  E  ASSESSMENT 


i 


Mission  &  Task  Driven 
Adaptive  Aiding 


Models  of  Cognitio  si.  Performance  and 

^  ^  PHYSlt)LOG' 


Neurally  Informed  Displays 
with  Individual  Differences 


I 


A 

I 

I 


Natural  User-System  Interactions:  Trustworthy  Proactive 


biological,  Behavioral,  And  Cognitive  Sensing 
&  Assessment 


INTERFA^S 


S&T 

Focus 


•• 


Co4nition,  Performance  and  Individual 

Differences 


•  ••  Socially-GuIded  Machine  Learning 


I 


•  • 


4 


_Computatio 
In 


I 

nal  Models  of  Operators’  Beliefs,  Desires, 

ENTIONS  AND  OTHER  MENTAL  STATES 


Molecular  Signatures 


Applied  Neuroscience 


•  • 


Gesture/non-verbal  interaction 


Human-System  Co-Adaptation 


is 

i 

4 


Shading  Legend 
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Intelligent,  Adaptive  Aiding 

Program  Detail 


S&T  Focus  Area 


Near-term 

FY  15 

FY  16 

FY  17 

FY  18 

FY  19 

Mid  /  Far-term 


Operational  Opportunities 


Gesture/Non-Verbal  Interaction 


Gesture  and  Non-verbal  Interaction 

© 

Brain-Computer  Interaction 

H 

Applied  Adaptive  Aiding 

twufi 

V 

Human-machine  interaction  using 
gestures  and/or  other  non-verbal 
means  to  communicate/execute 
mission  intent. 


Applied  Computational  Neuroscience 


Applied  Neuroscience 


Monitoring,  Predicting,  and  Optimizing  Battlespace  Awareness 

'  v 


Molecular  Signatures 

Soldier  Focused  Neurotechnologies 


Real-time,  omnipresent-sensing 
technology,  signatures  of  brain 
networks  that  capture  changes  in 
task  performance  and  brain-based 
technologies  to  aid  the  operator  and 
optimize  team  performance. 


Cognition,  Performance,  and 

Individual  Differences 


Cognition,  Performance  and  Individual  Differences 


Cognitive  Performance  Optimization 


0 

v 


Soldier  Centered  Design  Tools 


Advanced  technology  to  sense, 
measure  and  quantify  individual 
warfighter  cognition  and 
performance  parameters  to  predict 
and  augment  warfighter 
performance. 


Applied  Computational  Neuroscience 


Physiological,  Behavioral,  and 

Cognitive  Sensing  and 

Assessment 


Soldier  Focused  Neurotechnologies 


0 


Continuous  Multi-faceted  SoldierCharacterization  for 
Adaptive  Technology  Advancements 


Molecular  Signatures 


Cognitive  Performance  Optimization 


iA££liedAda£tiv^idin^ 


v 

V 

V 


On-line  operator  monitoring  and 
assessment  technology,  integrating 
multiple  and  concurrent  data  streams 
to  predict  and  augment  warfighter 
performance. 
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Protection,  Sustainment,  and 
Warfighter  Performance 
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HUMAN  SYSTEMS  COI  SUB-AREA: 

Protection,  Sustainment,  and  Warfighter  Performance 


VISION 

Enable  superiority  of  Warfighters  by  understanding  and  overcoming  operational 
stressors,  and  providing  protection  from  threats  in  their  environment. 


This  will  be  achieved  through:  Wearable  sensor technol°sy 

1 .  Understanding  the  factors  that  influence  individual  performance 

2.  Developing  the  ability  to  measure  performance  in  the  operational  environment 

3.  Developing  strategies  to  mitigate  the  effects  of  critical  stressors  on  performance 

Achieving,  this  vision  will  enable 

1 .  Increased  ability  to  perform  at  a  higher  stress  level  without  a  performance  decrement  or  increase  in  injury 

2.  The  ability  to  measure  performance  in  training  and  operational  environments 

3.  Warfighter  protection  aligned  to  mission  specific  threat,  environment,  and  region  allowing  for  optimal 
performance  while  maintaining  protection 

4.  New  technology  capable  of  measuring  current  Warfighter  state  and  predicting  current  and  near  term 
performance,  resulting  in  20%  increase  in  task  performance 

5.  Load  mitigation  strategies  resulting  in  25%  decrease  in  metabolic  cost 
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Protection,  Sustainment,  and  Warfighter 

Performance  Scope 


Research  and  development  in  this  area  will  produce  better  understanding  of  the  critical 
environmental  stressors  and  the  human  factors  yielding  individual  performance  differences 
in  operational  environments  in  order  to  enhance  performance  and  mitigate  the  effects  of 
stressors.  This  includes  designing  systems  that  support  and  exploit  individual  differences, 
and  developing  operationally  relevant  metrics  to  monitor  and  assess  performance. 


Thrust  Area  1 : 

Understanding  and  Quantifying 
Warfighter  Variability 

S&T  Focus  Areas  on  Roadmap: 

•  Ability  to  Conduct  Warfighter 
Assessment  in  All  Environments 

•  Mechanisms  and  Effects  of  Individual 
Differences  and  Critical  Stressors  on 
Warfighter  Performance 

•  Real-Time  Data  Analysis  and 
Performance  Prediction 


Thrust  Area  2: 

Enhancement  and  Mitigation  Strategies 

S&T  Focus  Areas  on  Roadmap: 

•  Tool(s)  for  conducting  trade  off  studies 
between  protection/load,  performance, 
and  individual  differences. 

•  Development  of  Augmentation 
Technologies  and  Techniques 

•  Design  and  Development  of  Models  and 
Methods  for  Understanding  Effects  of 
Mitigation  Strategies 
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Thrust  1:  Understanding  and  Quantifying 

Warfighter  Variability 


Delivering  the  Mission 

•  Data  analysis  and  performance  prediction  will 
enable  improved  resilience  by  providing  critical 
information  on  Warfighter  readiness. 

•  Understanding  the  underlying  mechanisms  through 
which  critical  stressors  influence  performance  will 
enable  greater  performance  and  protection 
methodologies. 

•  Understanding  individual  differences  in  the  effect 
of  critical  stress  on  performance  will  enable  greater 
Warfighter  resilience. 


Key  Technical  Challenges 


Delivering  Capability 

•  Developing  technology  capable  of  objectively 
measuring  warfighter  performance  in  operational 
environments  to  enable  real-time  monitoring  of 
Warfighter  performance. 

•  Understanding  the  underlying  mechanisms  through 
which  performance  is  influenced  will  provide  a 
pathway  to  optimizing  Warfighter  performance. 

•  Modeling  individual  responses  to  critical  stressors 
will  enable  the  leveraging  of  individual  variability  as 
a  means  of  improving  Warfighter  performance  and 
protection. 

Program  Overview 


Sensors  needed  that  are  non-invasive,  don’t 
influence  performance,  and  provide  meaningful  data 

The  underlying  mechanisms  by  which  specific 
stressors  influence  performance  are  poorly 
understood. 

The  influence  of  human  variability  on  the  effects  of 
stress  on  warfighter  performance  is  difficult  to 
predict. 

High  fidelity  models  that  predict  performance  and 
injury  and/or  the  impact  of  protection  strategies  on 
performance  are  lacking 


Determinants  of  hazardous  biomechanics 
Ubiquitous  and  unobtrusive  Real-World 
Assessment 

Impact  of  individual  differences 
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Understanding  and  Quantifying 
Warfighter  Variability 


2016 


2019 


I 

2021 


2023 


2026 
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Understanding  and  Quantifying 
Warfighter  Variability 


Near-term 

FY  15 

FY  16 

FY  17 

FY  18 

FY  19 

Mid  /  Far-term 


Operational 

Opportunities 


Physical  Performance  and 

Individual  Differences 

Understanding  the  effects  of 
physical  stress  and  of  individual 
variability  on  the  effects  of  that 
stress  on  performance. 


Bioeffects :toxic  particles,  nanomaterials,  directed  energy  exposures 


S^Effects  of  operational  environment  on  pilot  toxicology 


Human  Integrated  Performance  Optimizer 


An  understanding  the  individualized 
effects  of  critical  stressors  on 
physical  performance  will  enable 
greater  warfighter  resilience. 


Advanced  Research  focusing  on  Individual  Differences 


Real-Time  Data  Analysis  and 

Performance  Prediction 

Developing  the  ability  to  predict 
near  and  far  term  performance 
decrements  before  they  happen, 


-  High  resolution,  wearable  kinematic  sensor  and  real¬ 
time  algorithms  development 


©feedback  to  improve  Warfighter  Performance 
[Sustainment Technologies  for  Enhanced  Performance  of  Soldiers  (STEPS) 


Real-Time  information  on  Soldier 
state  and  impending  performance 
decrements  will  provide  critical 
information  on  Soldier  readiness. 


v  Real-Time  Bioeffects  analysis 


Bu' 


■  *  it  ■  n  ■ 


Arrays  for  Warfighter  Kinematic  Measurement 


Warfighter  Assessment  in  All 

Environments 

The  development  of  metrics  and 
tools  for  quantifying  Warfighter 
states  in  any  environment. 


mnipresent  Real-World  Soldier  Assessment 


©vs 


•v?  Aerospace  Toxicology  Human  on  a  Chi^ 
-r  .ntea  rated  SensorSuite  Developments 

Probabilistic  Risk AssessmentTools  forWeaoons  Developers 


The  ability  to  collect  information  on 
Warfighter  state  in  the  operational 
environment.  This  information  can 
be  used  to  prevent  performance 
decrements. 
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Pilot  Physiological  Monitoring  and  Warning 

System 

(PPMAWS)  Technology  Demonstration 


International  CRADA 
Elbit/LifeBeam  Helmet  Testing 


Foreign  Comparative  Test 
PPMAWS  integration  into 
Joint  Helmet  Mounted  Cuing 
System 


PPMAWS  Demonstration 
Altitude  and  High-G 
Acceleration 
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Thrust  2:  Enhancement  and 
Mitigation  Strategies 


Delivering  the  Mission 

•  Physical  augmentation  to  reduce  metabolic 
cost  by  up  to  25% 

•  Modeling  and  Simulation  tools  capable  of 
predicting  physical  stress  on  the  Warfighter 
to  within  5%. 

•  Optimized  load  configurations  and  route 
planning  leading  to  a  10%  reduction  in 
metabolic  cost  and  10%  increase  in 
operational  performance. 


Key  Technical  Challenges 


Delivering  Capability 

•  Develop  methods  of  lessening  the  effects  of 
critical  stressors  on  Warfighter  performance 

•  Understand  the  underlying  mechanisms  by 
which  physical  augmentation  and  protection 
technologies  affect  performance.  Set  system 
requirements. 

•  Provide  the  tools  (M&S,  route  planning,  etc.) 
necessary  to  understand  the  relationship 
between  new  technology,  mission  requirements 
and  operational  effectiveness. 


Program  Overview 


•  Tools  to  model  effects  of  augmentation  on 
physical  performance  and  injury  potential 
are  still  in  development. 

•  Route  planning  tools  require  high  fidelity 
models  of  human  physiological  response  to 
critical  stressors. 

•  Individual  variability  influences  the  extent  to 
which  physical  augmentation  can  mitigate 
physical  loads 


•  Lower  Extremity  motor  adaptations  to  actuation 

•  Effects  of  physical  augmentation  on  walking 
efficiency 

•  Enhanced  Technologies  for  Optimization  of 
Warfighter  Load 


Photo  property  of  MIT  Prof.  Hugh  Herr  75  Amherst  St.,  Rm.  El  4- 
374L,  Cambridge,  MA,  02139,  (t)  617-258-6574,  hherr@media.mit.edu 
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Enhancement  and  Mitigation 

Strategies 


Mission 

Need 

Military 

Capabilities 


Technical 

Goals 


S&T  Focus 


Improved  operational  performance  through  load  mitigation  technologies 

i  i  i 

i  i  i 

P hysical  Augmentation  Devices  /  Exoskeletons 


Develop  tools  and  tec! 


S  tools 


Warfighter  off-loading  technology 


nology  toMessen  effect  of  load, 
environment,  anc  terrain  oh  physical  stress 

AA 


t 


,  i 


Eic 


Develop  an  understanding  of  the  ejffcts  of  load  mitigation  on 
performance  and  howjtf  optimize  it 

A A  1 


Develop  better  contrJl  algorithms  for  flhyLical  augmentation  devices 


me 


I  |A 

i  i 

fianical  and  others)  caplblelof  predictin^efjects  of  pljysicial  load  on  indi\/fcillls 


I 


I 


I 


^  L  evelopment  of  PI  lysical  Augmentation  Dfe  Jces 


I 

Q  b  odeling  Effects  c  f  Mitigation  Strategies 

I  i 


F  educing  Effects  of  High  d  Environments 


••I 


educing  Physical  Load 


2016 
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2021 


2023 


Shading  Legend 
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Light:  Not/partially  funded 
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#  Navy 
Air  Force 
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Enhancement  and  Mitigation 

Strategies 

Program  Details _ 


Near-term 


FY  15 


FY  16 


FY  17 


FY  18 


FY  19 


Mid  /  Far-term 


Operational 

Opportunities 


Warrior  Web 


Development  of  physical 

augmentation 

Devices  designed  to  lessen  the 
effects  of  physical  load  on  the 
Warfighter 


$  Tactical  Assault  Light  Operator  Suit(TALOS) 


l  Lower  Extremity  Adaptations  to  J  oint  Actuation 


Human  Body  adaptations  to  physical  augmentation 


■The  Effects  of  Training  on  the  Efficacy  of  a  Physical  Augmentation  Device 
Ejj  Advanced  control  algorithms  for  enhanced  augmentation 


Increased  endurance,  decreased 
physical  fatigue,  improved 
performance. 


■“[Ankle  Exoskeletons  to  assist  Load  Carriage 


Modeling  effects  of  mitigation 

M&S  aimed  at  improving 
augmentation  devices  and  better 
understanding  their  effects 


J  oint  Biomechanical  Modeling  and  Simulation  Initiative 
B0  Enhanced  Technologies  for  Optimization  of  Warfighter  Load 

V*  3-D  Modeling  &  Spinal  Injury  Assessment 


V 


Advanced  Human  Whole-Body  Response  Model 


Augmentation  devices  that  are 
better  suited  to  the  user,  resulting  in 
increased  physical  performance, 
and  less  cognitive  decrement 
resulting  from  physical  fatigue 


Reducing  Effects  of  High  G 

Environment 

Efforts  aimed  at  reducing  the  effects 
of  high  G  environments  for  pilots 


W  Hypersonic  Escape 
v  NextGen  Escape  Systems  Concepts  for  P ilots 


1 e petitive  G -Loading  mitigation  for  Pilots 


Increased  pilot  performance  in  high 
G  environments,  decreased  injury 


Reducing  Physical  Load 

Technology  aimed  at  reducing  the 
physical  load  (actual  weight,  ‘easier’ 
terrain,  etc.)  a  warfighter  needs  to 
traverse. 


NSRDEC  Route  Planning  Tool 


Energy  Harvesting  BackPack 


®0 


Load  Carriage  /  Novel  Load  Mitigation  studies 


The  ability  to  reduce  Warfighter 
physical  load  while  maintaining 
capability  and  performance. 
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Accomplishments 


•  OBOGS  -  Mil  Standard  3050  developed  (Bi-service 
Air  Force/Navy).  OBOGS’  oxygens  systems  and 
their  standards  (operating  and  contamination 
standards). 

*  Multiple  wearable  robots  are  showing  reduced 
metabolic  cost  during  walking  (treadmill  and 
overground) 

*  Warfighter  variability  within  field  based  settings 

-  IMUs  from  lab  to  field,  now  instrumenting  Warfighters  with  IMUs  and  getting 
kinematics  and  more  in  depth  performance  metrics  in  the  field.  Providing  more  in 
depth  information  than  that  which  is  gained  solely  from  SME  opinion.  Think 
dashboard.  What  value  does  this  provide? 

•  Laser  Eye  Protection  ATD 
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Success  Story:  Autonomy  Research  Pilot  Initiative 
Realizing  Autonomy  via  Intelligent  Adaptive  Hybrid 

Control 


Operational  Challenge 


Autonomous  control  of  multiple  unmanned  systems 
for  military  operations 


Problem:  Current  fielded  systems  fall  far  short  of 
desired  advanced,  highly  reliable  autonomous 
cooperative  behavior 


Objective:  Increase  the  robustness  and  transparency  of 
autonomous  control  for  multiple  unmanned  systems 


Outcome:  Agile  and  robust  mission  effectiveness  across 
a  wide  range  of  situations,  and  with  the  many 
ambiguities  associated  with  the  “fog  of  war” 


S&T  Accomplishments 

□  Refined  tri-service  “Base  Defense”  challenge  scenario  to 
include  more  unexpected,  dynamic  events 

□  New  rapid  joint  human-machine  “Course  of  Action”  tool 

□  New  Task  Manager  capability:  system  workload  balancing 

□  IMPACT  “DoD  Virtual  Lab”  refined  (Year  2) 

□  1  operator  x  12  vehicles  (simulation) 

□  IMPACT  Year  2  full  system  evaluation  underway  with  8  op 
experts 

□  Co-development  of  R&D  at  ARL,  NRL  and  SPAWAR 

□  To  date,  over  30  S&T  publications  produced 


Return  on  Investment 


Affordability 

•  Reduction  in  logistics  footprint  for  equipment  and 
personnel 

•  Risk  Reduction:  Opportunities  to  transition  IMPACT 
technologies  to  other  DoD  programs 


Readiness 


Force  multiplier:  Autonomous  control  of  multiple  weapon 
systems  with  fewer  personnel 


_  _  -  SPAWAR 

ARL  V 


L- 
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Success  Story: 

Multi-Modal  Communication  Management  Suite 


Operational  Challenge 

Problem:  C2  operators  experience  a  high  volume  of 
perishable  voice  and  chat  communication  across  disparate 
systems  leading  to  high  workload  and  missed  messages. 

Objective:  Provide  comprehensive  communication 
management  software  that  improves  real-time  operator 
performance  and  workload  in  high  comm  situations. 

Outcome:  Net-centric  software  with  integrated  voice  and 
chat,  spatial  audio,  automatic  speech  recognition,  keyword 
\spotting,  communication  recording,  search,  and  playback. 


Immjio300000000  -  300  MHz  [piay^toglRevi^otcs  (JO  | 

1.0  0  Bl  A-  j  A-*  j  r|x1 

f  X  Hostile  *  IQlQiaBIQQ  Previous  Next 

"" - - 

Flag  lines  for 
later  review 


[12:,0.34]Eagle2:  eagle-02  sort  northern 

!I2:53:49|Eaglel:  eaqle-Ol  s  engaged  BULLSEYE  0  4-  7  /28  22  ,000  bogey  cobra  targeted  south 
group  BULLSEYE  0  >-  7  /21  28 ,000  track  east  bogey 


|L2:54K)l]Eaglel:  eagle-01  ID  hostile  hostile 
[L2:54:06]Eaglel:  eagle-Ol  fox-2  southern 
|12:S4:10]Eagle2:  eagle-02  fax-2  northern 

|12:54:20]Eagle3:  eagle  01  and  2  eagle-02  swept  5  0/3  eagle-03  still  spin  cold  3-4  come  left 

|12:54:24)Eaglel:  eagle  01  and  2  reference  2-  6-  0 

ll2:S4:32)Eagle1:  BULLSEYE 

1 12: 54:3*;::  <*g  lei:  eagle-01  gate 

tl£S4:40]Eaglel:  eagle-Ol  has  group  heavy 

|12:S4:42|Eaglc2:  eagle  02  same 

[12:S4:51|Eagle2:  eagle-02  target  group  BULLSEYE  0-5-6/21 25 ,000  cobra  declare 
|12:54:59|Eagle3:  roger  cobra  declares  BULLSEYE  0  6-0  /28  24 ,000  bogey 
|12:55:04|Eaglc2:  eagle  02  target  0-  5-  8  /23  24 .000  bogey 


STT 


[3]  mmcl 


Text-to-Speech 

to  listen  to  chat\^ 
messages 


[12:49:561v»c:  Cowboy  one  is  tally  leaders 

[12:50:01)v>c  Cowboy  two  is  tally  four  ship  offset  container  sorted  northern  trailer 
|12:S006]vic:  Cowboy  one  id  hostile 

[12:50:41]vic  Cowboy  two  s  out  of  range  northern  leader  nose  four  miles  trackin  east 
[12:50:48]vic  Cowboy  one  and  two  egress  one  five  zero 
|12:50:53]v»c  Cowboy  one 
[12:51:52]v»c  Cowboy  three  gate 

[I2:52:001v>c  Cowboy  three  cobra  picture  single  group  disregard  two  groups  range  five 
|12:52tf9)vic  leaders  bullseye  zero  four  zero  ten  twenty  four  thousand  track  east  bogey 
[12:52:13)vic  Cowboy  three  target  group  bullseye  zero  three  two  ten  four  targeted  twenty  four 
thousand  bogey 

|12:53:30Jvic  Cowboy  three's  engaged  bullseye  zero  six  zero  twenty  two  twenty  four  thousand  tally 
four  four  tally  lour 

[12:53:40|vic  Cowboy  three  id  hostile  hostile  fox  two  northern  trailer  fox  two  southern  trailer 
|12:53:46)vic  Cowboy  three's  ten  miles 


viper  1  &  2  formation  is  offset  container 


S&T  Accomplishments 

□  Software  prototype  with  licensed  patent  on  IP 

□  Lab  evaluations  showing  increased  key-word  spotting 
performance  and  reduced  operator  workload 

□  Integration  into  AFRL/FAA/N aval  Undersea  Warfare  Center 
research  testbeds 

□  Collaboration  with  Carnegie  Mellon  on  custom  speech 
recognition  models  for  FAA  and  Domestic  Event  Network 

□  Integration  and  operational  demonstration  at  Western  Air 
Defense  Sector 

□  Cooperative  Research  and  Development  Agreement  with 
Global  Flyte  to  tailor  for  emergency  response  scenarios 

Listen  to  past 


audio  by  selecting 
desired  text 


Radio 

communications 
are  transcribed 
using  ASR 


Highlight  Keywords 

-  create  alerts  to 
critical  words 

Speech-To-Text 

sending  chat 
messages 


Return  on  Investment 

Affordability:  Intellectual  Property 
protected;  software  based  on  open 
source  tools  and  message  protocols 

Readiness:  (TRL  4/5)  Concept 
demonstrated  in  laboratory  and 
operational  settings;  CRADAto 
improve  readiness  for  emergency 
response  domain 
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Success  Story:  Capable  Manpower 
Control  Station  Human  Machine  Interface  (CaSHMI) 


Operational  Challenge 


Problem: 


1)  Current  UxV  control  paradigm  is  manpower  intensive  with 
inconsistent,  proprietary  HMIs. 

2)  A  single  UxV,  vehicle-centric  HMI  metaphor  does  not 
scale  for  multi-UxV’s,  mission  management  &  emerging 
autonomy 

Objective:  Develop  a  Navy  Mission-centric  HMI,  that 
enables  “Parallel  management”  of  multiple  UxVs,  with 
intermittent  warfighter  engagement  and  will  scale  with 
expected  automation  and  technology. 


Outcome:  Transition  UxV  supervisory  control  HMI  & 
supporting  software  architecture  to  AN/BYG-1  Submarine 
/Ombat  System;  and  PMA-281  Common  Control  System 


S&T  Accomplishments 

■  User-Centered  Design  Process  completed  with  Cross- 
Domain  UxV  operators 

■  Innovative  software  architecture  defined  separating  vehicle 
control  from  business  logic  &  HMI  to  enable  scalable 
interface  implementations 

■  Live  demonstrations  of  AN/BYG-1  operators  using  CaSHMI 
to  provide  simultaneous  supervisory  control  of  a  Blackwing 
UAS  and  multiple  IVER  UUVs  concurrently. 


Return  on  Investment 


Affordability 

■  Reduction  in  manpower  requirements  for 
increasing  UxV  mission  employment. 

■  Reduction  in  training  costs  with  “common”  mission 
management  interfaces 

Readiness 

■  Enable  single  operator  management  of  2+  UxVs 
for  an  ISR  missions 


Flexible  task  management  supports  dynamic 
mission  events  /  requirements 


Prototype  for  Common  HMI  &  controls  across 
UxVs  &  Navy  platforms 
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Success  Story:  Medical  Technology  Research 
Closed-Loop  Oxygen  Generation  and  Delivery 


\ 

Operational  Challenge 

Closed-loop  control  of  oxygen  generation  and 
delivery  for  military  medical  operations 

Problem:  Current  military  environments  present  significant 
challenges  to  patient  care  in  operational  settings  (oxygen 
availability,  situational  awareness,  etc.) 


Objective:  Induces  automatic  changes  in  oxygenation 
delivery  during  mechanical  ventilation  in  response  to 
measured  changes  in  patient  physiology 


Outcome:  This  technology  has  the  potential  to  have  a 
profound  impact  on  the  way  the  military  medical  system 
cares  for  critical  care  patients 


Accomplishments 

□  Technology  has  demonstrated,  in  pre-clinical/clinical  models, 
successful  mitigation  of  hypo/hyperoxemic  events  (both 
associated  with  worsening  outcomes) 

□  Generated  novel  mechanical  ventilation/oxygen  concentrator 
interoperable  system 

□  The  research  team  has  received  an  FDA  Investigational 
Device  Exemption  (IDE)  to  conduct  a  first-of-its-kind  clinical 
trial  utilizing  closed  loop  control  of  oxygen  delivery  during 
mechanical  ventilation  in  trauma  patients 

^  \ 

Return  on  Investment 

This  technology  would  maximize  safe  oxygen  delivery 
and  minimize  oxygen/power  consumption 

Affordability 

•  Conserves  oxygen,  potentially  reduces  logistical 
planning  factors 

Patient  Safety 

•  Maintains  clinician  set  target  equivalent  to/or  more  often 
than  standard  of  care  (demonstrated  in  previous  trial) 

Readiness 


Force  multiplier:  Autonomous  control  of  multiple  patients 
with  fewer  personnel;  enhanced  care  of  wounded  in 
austere/resource  constrained  environments 
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The  Quantified  Warrior 


Multi-domain 

Operations 


Available  wearable  sensors  can  be  used  to  sense  the 
cognitive  and  physical  state  of  the  soldier,  sailor  or  airman. 
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Success  Story:  Optimized  Warfighter  Readiness 

Reduction  of  Sleep  Deprivation  Induced  Fatigue  Stress 


Operational  Challenge 


Identification  of  biomarkers  predictive  of  performance 
under  stress 


Problem:  Fatigue  is  an  important  concern  throughout 

today’s  24/7  operations  -  performance  degradations  cause  □ 

mishaps,  reduced  accuracy  &  slow  reaction  time. 

Objective:  Evaluate  efficacy  of  transcranial  direct  current 
stimulation  (tDCS)  to  reduce  cognitive  declines  caused  by  □ 
fatigue. 


Outcome:  Evidence  suggests  tDCS  is  twice  as  effective 
and  lasts  at  least  3  times  as  long  as  caffeine.  In  addition, 
test  subjects  report  feeling  less  fatigued  and  more 
energetic  24  hours  post-stimulation. 


S&T  Accomplishments 

Successfully  demonstrated  large  effects  of  tDCS  on 
cognition  and  mood  under  sleep  deprivation  conditions 

□  Evidence  suggests  tDCS  could  be  a  fatigue 
mitigation  tool  more  powerful  than  caffeine. 

Study  findings  have  been  replicated  within  AFRL  and 
outside  of  AFRL  laboratory 

□  Illustrates  effects  are  large  and  robust 
Developed  tDCS  paradigm  (electrode  placement,  current 
intensity,  stimulation  duration)  effective  for  stimulant-like 
effects 


A 


Return  on  Investment 
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Success  Story:  Human-Machine  Integration 
Advanced  Wearable  Technology  for 
Dismounted  Operators 


Operational  Challenge 

Dismounted  operators  require  greater  situational 
awareness  (SA)  and  an  integrated  tactical  ensemble. 

Problem:  Power/data  cable  hazards  and  responsive  multiple 
patient  monitoring 

Objective:  Increase  the  battlefield  airman’s  SA:  easily 
operate  and  increase  interoperability  of  BAO  &  GAO  kit 
components 

Outcome:  Medical  responsiveness  on  the  battlefield; 
minimize  operator’s  need  to  “look-down”;  and  easily  operate 
body-worn  devices 


Optical  Wireless 
Data  Connection 


BATDOK  (Battlefield  Airmen  Trauma 
Distributed  Observation  Kit) 


S&T  Accomplishments 

□  Developed  personal  area  optical  data  connection  to  link 
head-worn  devices  with  body-worn  devices  such  as 
headsets,  HMDs,  tablets,  radios,  etc. 

□  Developed  person-to-platform  optical  data  connection  to 
link  untethered  Airmen  to  mission  platforms  for  wireless 
communications. 


□ 

□ 


Developed  sensor/wireless  protocol  agnostic  casualty 
monitoring  application  and  system 
Developed  EUD  multimodal  covert/overt  dismounted 
notifications  alerting  medics  of  patients’  urgent  conditions 


Return  on  Investment 


Affordability 

•  Reduction  in  BAO  &  GAO  kit  cost  due  to  the 

elimination  of  cabling  needed  to  connect  with  body- 
worn  devices 


•  Casualty  monitoring  device  and  capability  can  save 
lives 


Readiness 

•  Remote  monitoring  of  multiple  casualties 

•  Ease  of  operating  body-worn  devices 

•  Reduction  in  cable  hazards 

•  Reduced  training  spin-up 
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Success  Story:  Cognitive  Assessment  Metrics  and 
Emerging  Reality  Augmentation  (CAMERA) 


Operational  Challenge 

Problem:  New  sensing  technologies  require  Warfighters  to 
make  accurate  decisions  based  on  a  myriad  of  data  while 
operating  in  chaotic  environments,  but  assessing  human  use 
of  Situational  Awareness  (SA)  technologies  in  standard 
mission  sets  has  not  been  formalized. 


□ 

□ 


S&T  Accomplishments 

Developed  scenarios  designed  to  variously  tax  cognitive 
workload  and  SA 

Developed  a  high-fidelity  PACOM  environment  with  local 
national  and  insurgent  behavioral  profiles,  realistic  weather, 
wildlife,  and  audio 


Objective:  Develop  validated  cognitive  workload  measures 
and  metrics  to  assess  the  impact  of  SA  technologies  on 
Soldier  cognitive  workload.  Develop  initial  standards  for 
cognitive  and  mission  performance  for  Dismounted  Soldier 
tasks  and  select  Mounted  Soldier  duty  positions. 


Outcome:  Increased  Situational  Awareness  during 
operational  missions  with  minimized  SA  technology  cognitive 
burden  on  Soldier  and  Small  units. 


Completed  pilot  studies  to  establish  test  methodology  for 
means  to  collect  physiological  metrics  such  as  voice  data, 
eye  movements/pupillometry,  and  electroencephalogram 
Approved  FY17  STO-R  to  develop  standard  documented 
test  bed  to  assess  impact  of  new  SA  systems  on  decision¬ 
making  and  workload  during  development  cycle 


Return  on  Investment 


Affordability 

Common  Dismounted  Soldier  viewing  and  computer 
control  experience  across  handheld,  mounted,  and 
thru-sight  displays  will  reduce  the  costs  of  developing 
related  technologies  and  training  personnel 


Readiness 

SA  technologies  deployed  more  rapidly,  with  fewer 
unknown  performance  drawbacks,  and  reduced  training 
time  due  to  validated  cognitive  assessment  and  common 
viewing  prior  to  fielding 
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Success  Story:  Computer  Generated  Forces 
Training  Executive  Agent  (TXA) 


Operational  Challenge 


“An  integrated  LVC  training  environment  with 
today’s  battle  complexity  is  essential  to  improving 
proficiency  across  all  current  and  future  mission 
sets.”  (Naval  Aviation  Vision  2016-2025) 


Problem:  High  manpower  to  run  complex  virtual  training 


Objective:  Make  Computer  Generated  Forces  (CGF)  more 
intelligent  and  adaptive  to  training  objectives 

Outcome:  Transitioned  the  Training  Executive  Agent  (TXA) 
into  the  Navy’s  Next  Generation  Threat  System 


S&T  Accomplishments 


■  TXA  monitors  a  training  exercise  and  issues 
“directives”  to  other  CGFs  to  modify  behaviors 
according  to  a  higher  level  scenario  director 
(training  objectives) 

■  TXA  used  in  NIFC-CA  training  scenarios 


■  Exploring  TXA  usage  on  aviation  pods,  thus 
providing  unique  flexible  embedded  training 
capabilities 


Return  on  Investment 


Affordability 

■  Aid  instructors  and  “pucksteers”  who 
dynamically  controls  CGF  during  execution  of 
a  training  scenario. 

■  Reduce  number  of  required  “pucksteers”, 
reduce  overall  training  costs 

Readiness 


■  Provide  trainees  with  tactically  realistic  entities, 
in  realistic  complex  battle  scenarios 

■  Allow  instructors  to  focus  on  trainee,  not  on 
playing  roll  in  scenario 
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Aircraft  Crew  Breathing  Systems  OBOGS 
MIL-STD  3050  -  Joint  Air  Force  Navy 


Oxygen  Technology  Evolution 


Gaseous  Oxygen  (GOX) 
Systems 


F-15E  On-Board 


Oxygen  Generating 
System  (OBOGS) 


Liquid  Oxygen  (LOX) 
Systems 


1920  1950 


1990 


F-22  On-Board 
Oxygen  Generating 
System  (OBOGS) 


Next  Generation 
OBOGS/ 
Solid  State 
OBOGS 


AlRCRAf  f  CKLYi  toftEAJHIWG  SYSILMS 
USING  ON  BOARD  OXYGEN  GENERATING 
SYSTEM  (060GS> 
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^  semi  Flex  8C 

SEMI  Strategic  Association  Partner 

Collaboration  for  Breakthrough  Innovation  in  Human 
Performance  Monitoring  for  the  Warfighter 

NDIA  2018  Human  Systems  Conference 


Dr.  Melissa  Grupen-Shemansk^ 

Chief  Technology  Officer,  SEMI  /  FlexTech 


SEMI:  Connect,  Collaborate,  Innovate 


R&D 

Consortia 


System 

Integration 


Application 

Market 


Device 


•  Collaboration  is  critical  to  solving  issues  and  innovating  new  products 

•  What's  needed  for  breakthrough  applications  is  almost  invisible  upstream  in  the  supply  chain 

•  Collective  focus  on  challenges  and  opportunities  shortens  time  to  market 


FlexTech 

SEMI  Strategic  Association  Partner 


SEMI  Technology 
Community 


SEMI 


Products  &  Services 

Industry  Events 


Industry  leadership  committees 


Funded  R&D 


Pilot  Manufacture 


s  e  m  i 


FHE  System  Schematic 


Flexible,  Hybrid  Electronics  (simplified) 


Processor 


Flexible 

Substrate 


#  Mostly  printed 

*  Can  be  printed 

+  Not  usually  printed 


Display 


Polymer  Solar  Cell  # 


Sensors  * 


Memory 


Antenna# 


Thin  Film  Battery#  Communications 

Interface  * 


Source:  FlexTech  Alliance 


Mostly  Printed 

•  Interconnect 

•  Antenna 

Can  Be  Printed 

•  Power 

•  Display 

•  Communication  Interface 

•  Sensors 

Not  Usually  Printed 

•  Processor 

•  Memory 

•  Medium  to  far  field 
communications  and  location 

•  High  performance  circuits 


FlexTech 

SEMI  Strategic  Association  Partner 


SEMI  •  FlexTech  Activities 


Mission:  Create  a  collaborative  environment  to  accelerate  the  risk 
reduction  of  technology,  manufacturing,  and  supply  chain  development 


Flexible  Hybrid 
Electronics  Supply  Chain 
Development 


FlexTech 


Flexible,  Wearable 
Human  Performance 


Monitoring 


•  CMOS  Integration 

•  Radio  and 
communications 

•  Sensing,  warnings, 
wearable  displays 


•  Human  Performance 
Monitoring  Applications 

•  20+  members 


Air  Force 
Research 
Laboratory 


Flexible  Hybrid 
Electronics 

Manufacturing 


N  EJTT  F  L 

America’s  Flexible  Hybrid  Electronics  Manufacturing  Institute 


•  Manufacturing  Gaps 

•  Public-private 
partnership 

•  50+  members 


y:  Manufacturing* * 

Lusa 


FlexTech 

SEMI  Strategic  Association  Partner 


Projects  &  Partners 


Sensor  Systems  Materials  Power  Hybrid  Integration 


Design  &  Integration 


m  i 


chm 


molex 
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THE  UNIVERSITY 

of  Arizona, 


Biometric  Sensors  - 
Integrated  Development 
Platform  for  Human 
Performance  Monitoring 


I  SIP 


Printed  Microfluidic 
Performance  Assessment 
for  Sweat-Based  Biomarker 
Sensor  Platforms 


ffiCHM 


A  Scalable  Flexible 
Substrate  and 
Assembly 
Process 


UCLA 


Flexible  Printed 
Electronic  Device 


molex 


Materials  Registry 


"jyiatWeb 


Flexible 

Electromagnetic 
Field  Sensing  Array 
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PRINCETON 

UNIVERSITY 


Self-Powered 

Communicating 

Sensors 


Flexible  Printed 
Battery 


Berkeley 

UNIVERSITY  OF  CALIFORNIA  J 
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Thin  Film  Power 
Source 


CUSTOM  ELECTRONICS,  IN 


Solid  State  Thin  Film 
Lithium  Rechargeable 
Battery 
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FlexTech 
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Flexible  Silicon 


American  Semiconductor  Inc. 


molex 


Ultra-thin  CMOS 

Assembly  &  multi-layer  flexible  PCB 


CMOS  wafer  prep 


Program  supported  by: 


Pick  &  Place 

25  micron  total 

packaged 

thickness 


one  company  >  a  world  of  Innovation 


3D  Additive  Interconnection 
additive  print  interconnect 


Roll  to  Roll  Manufacturing 
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FlexTeqh 

SEMI  Strategic  Association  Partner 


Wearable  Performance  Monitor 

Objective:  Develop  Flexible,  Non-invasive  Wearable  for  Dynamic  Assessment  of 

Hydration  Status 


Readout,  Analysis  & 
Communications  Hardware 


Pickup  Coil 

Barrier  RFID 
Film  Tag 

Adhesive 


Absorptive  Valve 
Pad 


Capillary 

Channel 


Wick  Vent  Perforations 

Air 
Skin 


NBMC  Consortium  Project  16-10:  A  Collaborative  team  of  7  industry  and 
university  members  lead  by  GE  developed 

1.  Ion  Selective  Electrodes  -  Na+  and  K+  concentration  in  sweat 

2.  RF  Impedance  Patch  -  subcutaneous  hydration  tomography  and 
spectroscopy 


s  e  m  i 


Program  supported 
by: 


FlexTech 

SEMI  Strategic  Association  Partner 


Thin  Flexible  Power  Source 
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Self-recharging 
Ultra-thin 
Power  Pack 


1TN  Energy  Systems  Inc. 
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Power  Module  Integration  +  Advanced  Materials  +  Photovoltaics 
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Imprint 


200 


Primary  &  rechargeable  battery 
Capacity  30-90  mAh 


Adv.  Packaging 
Integration 


Printing  & 
Lamination 
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Primary  battery 
Capacity  30-1000  mAh 


400  600  800 

Packaged  Battery  Thickness  (microns) 


1000 


Program  supported  by:  s  MKL 
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Medical,  Health  and  Wellness  Applications 


Cognitive  Function 


•  Military,  consumer, 
industrial,  and 
athletics 


•  High  value  assets, 
safety,  performance 


s  e  m  i 


•  Vital  sign  and  geriatric 
patient  monitoring 

•  Reduced  health  care 
costs 

•  Continuous 
measurement 
capability 


Treatment  Response 


Reduce  treatment 
cycle  times 

Reduced  costs 

Lower  mortality  rates 


•  Coordinated  triage 

•  Continuous  vital  sign 
monitoring 

•  Variable  /  austere 
environment 


Performance 

Monitoring 


•  Improved  health  and 
wellness 


•  Athletic  performance 
enhancement 


Monitoring  Requirements 

•  Cost  effective 

•  Unique  accuracy  &  precise 

•  Low  maintenance 

•  Automated  analytics 


SEMI  Strategic  Association  Partner 


Summary 


•  Target  the  needs  of  the  military  -  sustaining  and  augmenting  peak  performance  of 
the  military  personnel 

•  Align  to  commercial  industry  advances  and  ecosystem  development  to  ensure  a 
stable  and  advanced  supply  chain 

•  Take  advantage  of  other  disruptive  technologies  and  address  the  challenge  of 
incorporating  into  flexible  electronics 

—  Artificial  intelligence  and  machine  learning 

—  Edge  computing  and  new  SW/HW  architectures 

—  Advanced  MCU-memory-sensor  interfaces 

—  Multi-modal  sensor  data  acquisition  and  management 


FlexTech 

SEMI  Strategic  Association  Partner 


Thank  You 


Flexlech 

SEMI  Strategic  Association  Partner 
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DHS  SCIENCE  AND  TECHNOLOGY 


HSI  T&E  Methods  and  Metrics  for 
Assessing  User-Automation  Interaction 


13 -14  March  2018 


Homeland 

Security 

Science  and  Technology 


Mike  Barrientos 

Transportation  Security  Laboratory 

Bonnie  Kudrick 

TSA  Office  of  Requirements  and  Capabilities  Analysis 

Janae  Lockett-Reynolds,  Ph.D. 

Office  of  Systems  Engineering 

Thomas  Malone,  Ph.D. 

Carlow  International,  Inc. 


Agenda 


•  Definition  of  Automation 

•  Allocation  of  Functions 

•  Value  of  Early  T&E 

•  Systems  Engineering  Life  Cycle  (SELC) 
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Automation 


Technology  by  which  a  machine  performs  a  process  or  procedure 


Advantages 


\uiin\ihmitiivuir 


Disadvantages 


•  Reduces  human  workload,  fatigue,  staffing 

•  Integrates  large  volumes  of  data 

•  Improves  vigilance,  surveillance,  and 
detection 

•  Increases  safety  by  removing  the  human  from 
hazardous  environments  or  situations 

•  Reduces  human  error 

•  Expands  and  extends  human  capabilities: 
strength,  sensory  abilities,  and  cognitive 
processes 


•  Uses  different  control  strategies  than 
operator 

•  Elicits  the  ‘out  of  the  loop  syndrome’ 

•  Leads  to  complacency 

•  Leads  to  loss  of  perishable  skills 

•  Creates  mode  selection  errors 

•  Interferes  with  operator  situation 
awareness  issues 

•  Interferes  with  team  coordination 
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Function  Allocation 

in  the  system  design  process 


Decisions  are  made  on  functions: 

•  which  ones  are  fully  automated 

•  which  ones  are  performed  by  a  human  only 

•  which  ones  are  performed  by 

a  collaboration  between  automation  and  human 


Criteria  for  these  decisions  include: 

•  technology  maturation,  technical  risks, 
and  automation  feasibility 

•  command  authority,  mission  risks, 
environmental,  and  operational  contexts 

•  human  safety,  performance  capabilities, 
and  limitations 
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Value  of  Early  T&E 


Initiating  T&E  efforts  earlier  in  the  process  leads  to: 

•  Successful  T&E  outcomes  through  early  verification  of  HSI  requirements; 

•  Early  identification  and  implementation  of  mitigation  strategies  to  reduce  HSI  risks; 

•  Earlier  identification  of  problems  that  can  impact  system  modifications; 
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DHS 

Systems  Engineering  Life  Cycle 


Mission 

Analysis 


Early  HSI  Assessments 


Need  Phai 


Needs 

Analysis 


Produce/Oeploy.'Support  Phase 


Technology  Development 


❖  ♦ 


Disposition 
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Key  HSI  Activities 
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Sample  HSI 

Methods  and  Measures  for  Automation 


Methods 

Measures 

Metrics 

Direct  observation  and 

Usability 

Completion  rate,  number  of  incorrect  decisions/actions,  duration  of  time  to 

measurement 

respond  to  a  stimulus  or  complete  a  task,  ratings  of  satisfaction/acceptance) 

“^Experimentation 
**Actual  operations 
**Simulated  Human-in- 
the-loop  studies 

Workload 

Number  of  instances  where  task  load  exceeds  threshold,  duration  period  of 

(e.g.,  task  load/task  engagement) 

workload  saturation,  workload  spikes/number  of  instances  or  timing  situations 
where  the  overall  workload  drops  below  a  set  minimum  threshold,  ratings  of 
perceived  workload 

Training 

(e.g.,  manual  and  automated  tasks) 

Time  to  train/number  of  training  objectives  met 

Compliance  with  standards 

Mismatches  between  authority  and 
responsibility 

Number  of  mismatches  between  authority  and  responsibility 

Heuristic  Evaluations 

Unpredictability 

Number  of  Type  1  (i.e.,  human  cannot  predict  the  need  to  act  in  nominal 
operations)/Number  of  Type  2  (i.e.,  human  can  predict  but  the  timing  is  uncertain 
regarding  when  the  action  needs  to  occur)  unpredictable  actions 

Checklists 

Subjective  rating  scales 

Coherency 

Number  of  resource  conflicts  resulting  in  interdependent  activities/significant 
coordination 

Surveys/Questionnaires 

Interruptions 

Number  of  interruptions 

Automation  boundary  conditions 

Number  of  instances/duration  when  automation  is  placed  outside  of  boundary 
conditions.  Number  of  instances  where  automation  does  not  achieve  targets  while 
operating  according  to  its  spec 

Adaptation  to  context 

Number  of  instances  where  there  is  an  Inconsistent  response/work-arounds  to 
overly  prescribed  function  allocation 

Automation  reliability 

Ratings  of  trust/degree  of  overreliance  on  automation/degree  of  disuse 

Transfer  of  control 

Duration  of  time  it  take  to  move  from  a  passive  to  an  active  state/#  of  errors 

Situation  awareness 

Mean  Situation  Awareness  Global  Assessment  Technique  (SAGAT)  scores  across  a 
number  of  simulation  trials 

Comprehension  of 
system/automation  logic 

End  user  rating  of  the  extent  to  which  the  automation  meets  expectations 

T&E  Activities:  Solution  Engineering 


•  Identify  applicable  HSI  MOSs  and  MOPs 

•  Evaluate  alternative  concepts 

o  Develop  test  scenarios 

o  Select  HSI  methods  to  assess  function 
allocation  approach  and  define  roles  of 
human  and  automation 


•  Identify  HSI  operational  requirements  and 
critical  operational  issues 


Analyze/Select  Phase 


Operational  Requirement:  The  solution  shall  enable  users  to  successfully  complete  mission  tasks 
with  reduced  workload  (Threshold  =  X  and  Objective  =  X  ) 

COI:  Does  the  design  of  the  automation  coupled  with  user  training  support  the  users  in  successfully 
accomplishing  mission  tasks  and  minimize  the  deskilling  of  automated  tasks? 


T&E  Activities:  Planning 


•  Provide  HSI  inputs  to  the  TEMP 
(Test  and  Evaluation  Master  Plan). 

o  HSI-specific  tests 

o  HSI  tests  as  part  of  system-level 
testing 

•  Develop  an  HSI  test  strategy 

•  HSI  participation  in  a  T&E  IPT 

•  Provide  HSI  Critical  Technical 
Parameters  (CTPs) 


Obtain  Phase 


Development 

(R&D) 


HSI  Measures: 

Critical  Technical  Parameter: 

Usability 

Situation  Awareness 
Coherency 

Task  completion/error  rates  for  task  work/teamwork, 

Mean  SAGAT  scores 

Number  of  resource  conflicts  due  to  overlap  in  functions 

T&E  Activities: 
Requirements  Definition 


•  Determine  how  HSI  T&E  activities  will  refine 
requirements. 

•  Determine  how  HSI  T&E  is  used  to  assess 
the  adequacy  of  commercial  or  COTS 
products. 


Obtain  Phase 


Requirements 
Definition 
(Functional  Design) 


•  Refine  requirements  through  experimentation 
and  modeling  and  simulation. 


SDR 

Integration 
and  Test 


Requirement:  System  function  allocation  concept  shall  support  the  dynamics  of  the  work 

Experimental  Design:  2X2X3  (12  conditions) 

IVs:  allocation  concept,  experience  level,  task  demands 

DVs:  time  spent  in  cognitive  control  modes  and  transitions  between  control  modes 


T&E  Activities:  Design 


•  Develop  HSI  inputs  to  the  DT&E  test  plan 

•  Common  methods  (analysis,  test, 
demonstration,  inspection) 

•  Iterative  testing  and  evaluation  to  assess 
user  performance  of  human-automation 
interaction 

o  Usability,  situation  awareness,  workload,  function 
allocation  concepts 


Obtain  Phase 


Design 

(System  Design) 


Design 

Reviews 

Integration 
and  Test 


Requirements: 

System  shall  provide  adequate  early  warning  notification 

System  shall  not  interrupt  at  inappropriate  times  (e.g.,  periods  of  high  workload) 

System  shall  enable  the  user  to  determine  asset  health,  status,  and  place  in  a  procedural  sequence 


T&E  Activities:  Development 


HSI  testing  of  components,  products,  and 
functionality  that  make  up  the  system 

o  Evaluate  human  performance  against 
TPMs  to  assess  progress  towards 
meeting  system  performance 
requirements 

o  Identify  HSI  risks  and  mitigation 
strategies 


Obtain  Phase 


Development 


IRR 


Integration 
and  Test 


HSI  Risks: 

•  If  automation  overly  prescribes  a  course  of  action  or  sequence  of  activities,  then  users  may 
establish  work-arounds  or  stop  using  automation 

•  If  authority  for  task  execution  and  responsibility  for  the  outcome  are  not  assigned  to  the  same 
“agent”,  then  increased  workload  due  to  monitoring  associated  with  the  mismatch  may  occur 


T&E  Activities: 
Integration  and  Test 


Determine  if  integrated  system,  subsystems,  and 
components  satisfy  HSI  requirements  (DT  and  OT): 


Coordinate  with  OTA  so  that  Early 
Operational  Assessments  address 
progress  toward  meeting  HSI  COIs 

Assess  system  risks  and  maturity 

Identify  and  implement  risk  mitigation 
plans  and  corrective  actions 


Assess  human-automation  collaboration  effectiveness  &  efficiency  in  the  completion  of 
mission  tasks  in  representative  environments  using  engineering  simulations  with 
prototypes  and  actual  systems 


In  closing... 


•  There  are  benefits  to  “shifting  left”  and  assessing  HSI  considerations  earlier  in 
the  acquisition  process. 

•  T&E  must  evaluate  decisions  concerning  verification  of  type/level  of 
automation. 


Modern  DHS  systems  increasingly  exhibit  automated  processes  (security 
network  monitoring,  revenue  collection  systems,  explosive  threat  detection 
systems,  remote  video  surveillance  systems,  etc.) 

Unique  opportunities  exist  for  designing  automated 
technology  and  systems  that  facilitate  teamwork 
and  cooperation 
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Questions? 


Science  and  Technology 
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HSI  "&E  -  Typical  Measures 


•  MOEs  (Measures  of  Effectiveness)  are  derived  through  an  analysis  of  the 
mission  by  initially  defining  the  top-level  mission  tasks  (capabilities)  required 
to  be  performed,  regardless  of  the  potential  solutions. 

Typically,  human  performance  is  not  addressed  in  MOEs. 


•  MOSs  (Measures  of  Suitability)  are  critical  operational  measures  of  success 
that  define  whether  the  alternative  is  a  suitable  solution.  MOSs  include  items 
such  as  the  impact  of  the  alternative  on  the  users  (e.g.,  human  system 
integration,  accessibility,  compliance,  privacy,  tactics,  training)  or  the  public 
(e.g.,  flow  of  commerce,  environmental  impact)  as  well  as  sustainability  and 
support  requirements. 


•  MOPs  (Measures  of  Performance)  are  specific,  technical  performance 
parameters  that  can  be  measured  and  contribute  to  the  MOEs/MOSs. 
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Integrity  *  Service  *  Excellence 


Developing  an 
Autonomous  Task 
Manager  for  Intelligence, 
Surveillance,  and 
Reconnaissance  Human- 

Machine  Teams 

NDIA  Human  Systems  Conference 

3/11/18 

Jennifer  Lopez,  MA 
Mary  Frame,  PhD 
Alan  Boydstun,  PhD 
Analyst  Test  Bed 
711th  Human  Performance  Wing 
Air  Force  Research  Laboratory 

yv 
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Current  Analysts’  Challenges 
Within  ISR  Environment 


•  Workload  consists  of  multiple  tasks  and  long  shifts 

•  Work  within  Human-Machine  Teams  (HMT)  to  Process, 
Exploitation  and  Dissemination  (PED)  Essential  Elements  of 
Information  (EEls)  to  customer  down  range 

•  Multitasking  environment  may  require  automation  to  improve 
overall  performance  (accuracy 

and  efficiency) 

-  Reduce  repetitive  “busy  work” 

Examples: 

Copy  &  Pasting 
Target  Detection 
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Initial  Autonomous  Manager  (AM) 


Autonomous  Manager  (AM)  is  a  “new  agent”  within  the  PED 
cell.  Through  simulation,  AM  currently: 

•  Intelligently  and  dynamically  parses  task  allocation  in  real¬ 
time  based  on  agent  performance  and  workload 

•  Simulates  performance  with  varying  prior  uncertainty 

•  Incorporates  physiological  indicators  of  human  workload 
(e.g.  heart  rate) 


(a) 


Human  73% 


Automation  62% 


(J3^)  Automation  34% 


(b) 


(a)  Multi-INT  dashboard  of  four  tasks  (e.g.  Tl,  T2,  T3  and  T4) 

(b)  Example  of  task  allocation  based  on  performance  criteria  and 

workload 
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AM  Simulation  Performance 


Performance  Across  Configurations 
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Mean  Performance 

•  AM  Parser  performs  11 .37%  better 
than  Human  baseline 

•  Optimal  Performance  performs 
5.46%  better  than  AM  Parser 


AM  Parser  Improvement  Over  Human 
Operator  Alone 
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Performance  Improvement 
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Performance  Improvement  (A) 

Under  which  task  conditions  do  we 
find  greatest  and  lowest  degree  of 
improvement 


88ABW-2018-0978  28  February  2018 


£ 


4 


AM  with  Nonstationary 
Performance 


Performance  Across  Nonstationary 
Configurations  (1  change) 


1000 

900 

800 

700 

600 

500 

400 

300 

200 

100 

0 


L 


0.1  0.2  0.3  0.4  0.5  0.6  0.7 

Performance  (Accuracy) 

■  Optimal  ■  AM  Parser  Human 


0.8  0.9 


Performance  Across  Nonstationary 
Configurations  (3  changes) 


1400 
1200  i 

1000 

g- 

c  800 

o 

a 

g  600 

u. 

400 

200 

0 


0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9 

Performance  (Accuracy) 

■  Optimal  BAM  Parser  Human 


Performance  Across  Nonstationary 
Configurations  (6  Changes  -  Random  Interval) 


5000  - 
4500 
4000 
3500 
g  3000 

|  2500 

£  2000 

1500 

1000 

500 

0  -- 


0.1  0.2  0.3  0.4  0.5  0.6  0.7 

Performance  (Accuracy) 

■  Optimal  BAM  Parser  Human 


Mean  Performance  of  Optimal 

and  AM  Parser 

Stationary:  5.46% 

Nonstationary  (1  change):  5.52% 
Nonstationary  (3  changes) :  6.12% 
Nonstationary  (6  changes) :  7.03% 


w 
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AM  Parser  Improvement  Over  Human 
Operator  -  6  Changes  Random  Timing 


Performance  Improvement 


Performance  Improvement  (A) 

-  Stationary:  11.37% 

-  Nonstationary  (1  change):  11.20% 

-  Nonstationary  (3  changes) :  10.79% 

-  Nonstationary  (6  changes) :  10.57% 
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Conclusions 


•Initial  simulation  shows: 

-  Performance  improvement 

-Robust  to  dynamically  changing  performance  conditions 

-Can  be  improved  with  more  sophisticated  models  of 
workload  and  more  flexible  performance  thresholds 
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Future  Work 


•Add  more  physiological  indicators  of  human 
workload 

•  Extend  AM  beyond  static  thresholds 

-  Physiological  &  Performance 

•  Model  hierarchical  dependences  between  tasks 
-Single  HMT,  Teams  of  HMTS 

•  Develop  task  environment 

-Modify  AM  to  real-time  parsing  of  physiological  and 
performance 
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Questions? 


Nonstationary  Mean  Performance 


Nonstationary  Summary  Statistics 

Condition 

Optimal  Mean  (SD) 

AM  Parser  (SD) 

Mean  Difference 

t 

P 

Effect  Size 

Nonstationary  (1  Change) 

66.81%  (11.92%) 

61.29%  (9.45%) 

5.52% 

58.16 

0.000 

0.513 

Nonstationary  (3  Change) 

66.97%  (11.99%) 

60.85%  (6.64%) 

6.12% 

56.07 

0.000 

0.631 

Nonstationary  (6  Change) 

67.04%  (11.91%) 

60.01%  (5.63%) 

7.03% 

57.19 

0.000 

0.755 

Condition 

Optimal  Mean  (SD) 

AM  Parser  (SD) 

Mean  Difference 

t 

P 

Effect  Size 

Nonstationary  (1  Change) 

50.09%  (14.53%) 

61.29%  (9.45%) 

11.20% 

80.67 

0.000 

0.914 

Nonstationary  (3  Change) 

50.06%  (14.62%) 

60.85%  (6.64%) 

10.79% 

74.59 

0.000 

0.948 

Nonstationary  (6  Change) 

49.75%  (14.77%) 

60.01%  (5.63%) 

10.26% 

67.63 

0.000 

0.918 
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Overall  Goals 


AKL 


Addressing  research  problems  relevant  to  transitioning  robots 
from  Tools  to  Teammates 


Vision:  Natural  and  intuitive  bi-directional  communication 
•  Spoken  dialogue  with  mixed  initiative 


Tool  Teammate 
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H:  Enter  and  scan  first  room 


R:  I  see  a  door  to  the  right  and  a  door 
to  the  left 

H:  Scan  next  open  room  on  left 


USC  Institute  for 
Creative  Technologies 

Virtual  Human 

(DeVault  etal.,  2014) 
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The 
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Project  Objectives 


AKL 


•  Understand  human  language  use  to 
design  robot  capabilities  needed  for 
effective  communication 


•  Collect  data  containing  natural  language 
commands  and  associated  multimodal 
data  from  robot 

•  Task:  Human  “teammate”  is  engaging 
in  dialogue  with  robot  “teammate” 


•  Can  we  successfully  leverage  dialogue 
management  approaches  from  Human- 
Virtual  Human  (VH)  dialogue  and  apply  to 
robots? 

•  Phased  “Wizard  of  Oz”  methodology 


Team  Members 

ARLCISD:  M.  Marge,  C.  Bonial,  C.  Hayes,  S.  Lukin,  C.  Voss 

ARL  HRED:  S.  Hill,  A.  Evans,  A.  Foots,  K.  Pollard 

ICT  (Army  UARC)@USC:  D  Traum,  R  Artstein,  A  Leuski 
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Bot  Language  Lifecycle 


AKL 


Phase  1  Exploratory  Data  Collection 
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Lifecycle  of  Human-Robot  Dialogue 
Development 


AKL 


Phase  1  Exploratory  Data  Collection 


Phase  3 
Full  Automation 
Of  “Wizard” 


Phase  2 

Automate  Some 
“Wizard”  Labor 
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Approach 
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(Marge  et  al.,  201 6,  IEEE  RO-MAN)  6 
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•  Dialogue  Manager  (DM- 
Wizard)  is  the  “brains”  of 
the  robot  in  natural 
language  interactions 


•  Robot  Navigator 

(experimenter)  navigates 
robot  based  on  instructions 
from  DM-Wizard 


VERBAL 

COMMANDS 

“Behind  the 
scenes” 
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Approach 


ARL 


Dialogue  Manager  (DM- 
Wizard)  is  the  “brains”  of 
the  robot  in  natural 
language  interactions 


Robot  Navigator 
(experimenter)  navigates 
robot  based  on  instructions 
from  DM-Wizard 
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Commander:  Speaks  into  microphone 

Requests  and  views 
photo  from  robot 


Reads  DM-Wizard 
dialogue  responses 


(10:18:33)  robot:  I’m  not  sure.  I  haven't  detected  any 
people  but  we  haven't  fully  explored  the  space. 
(10:18:37)  robot:  executing... 

(10:18:40)  robot:  done 
(10:18:46)  robot:  executing... 

(10:18:53)  robot:  I  could  only  move  13  feet. 
(10:19:09)  robot:  executing... 

(10:19:13)  robot:  Our  bandwidth  will  currently  not 
support  the  video  feed. 

(10:19:15)  robot:  Would  you  like  me  to  send  a  photo? 
(10:19:19)  robot:  sent 
(10:19:23)  robot:  executing... 
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Commander  View  am. 
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DM-Wizard  View:  Experiment  1  Ahl 


Dialogue  Manager:  Listens  to  Commander  via  headphones 


Views  video  from  robot 

II I 


Views  photo  sent  to 
Commander 


Views  real-time  map  +  robot 


To  Commander  To  Navigator 


C  JmiIMho* 

Cnatroom,  prooml 


connect  |  crutroom  f/room2 


(13:55:53)  system  message:  robot  has 
joined  Ihc  session 
(13:56:33)  robot:  sent 


*1  Holla*  I  MI/MMl 


(13.55:53)  system  message:  wi7ard  has 
joined  the  session 


Experiment  1:  Types  responses  into 
chat  windows  to  Commander  and 
Robot  Navigator 
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u.z.  M/xivir  —  — 

RDECOM  Video:  DM-Wizard  in  Experiment  1  ARL 


1 

r  d  mn 

To  Commander 

To  Navigator 

Chatroom.  [/rooml 


u  ■;.>  ClMlIVtiifM 

connect  |  Chatroom.  |/room2 


connect  | 


kfaMIIIMItlir  liutnil 


INHIlmlll**  Iwl/rtlll 


(13:55:53)  system  message:  robot  has 
joined  the  session 
(13:56:33)  robot:  sent 


(13:55:53)  system  message:  wi7ard  has 
joined  Ihc  session 
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DM-Wizard  View:  Experiment  2  MliL 


Dialogue  Manager:  Listens  to  Commander  via  headphones 


Views  video  from  robot 


Views  photo  sent  to 
Commander 


Presses  buttons  on  graphical 
interface  to  produce  replies 


'  I 

IhMiMW’  | 

(13:55:53)  system  message:  robot  has 
joined  the  session 
(13:56:33)  robot:  sent 

(13:55:53)  system  message.  wi7ard  has 
joined  Ihc  session 

Experiment  2:  DM-Wizard  interface 
instead  of  typing 
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Video:  DM-Wizard  in  Experiment  2  Arl 


"Move  do  move  south" 


b  Front  Facing  Camera 


□  Last  Image  Sent 


I 


J  wiz-co 

4*  ©  fUe:///home/bbyrne/ros/cpr Jade_ws/src/command_put  C  ^  S 


*  ft 


Wiz-Co  mtnander 


“ 

Wiz-RN 

Rooms 

Hallways 

Alley 

Intro  1  -  hello 

lntro2  thanks 

lntro3-  lag  time 

lntro4- 

feedback 

IntroS-  help 

also,  ready 

ready 

map 

appearance 

tech  issues 

standby 

how  puah  totals 

task  complete 

processmg 

executing 

moving 

turning 

sent 

done 

done  sent 

action  done, 
photo  sent 

just  a  moment 

hmm 

hang  on 

and. 

but  . 

however 

hear  you 

calibrating 

calibration 

complete 

yes 

no 

ok 

ok  got  < 

reponse 

unsure 

correct 

donl  think  so 

think  so 

good  job 

thank  you 

hi 

sorry 

woops 

course 

unsw*  or  object 
•Waned  to 

unsure  object 
meant 

size,  position? 

descnbe 
another  way7 

unsure  where 
logo 

unsure  ol 
doorway 

unsure 

doorway  meant 

unsure  ot  room 

unsure  of  wall 

one  to  my 
nght? 

on  the  right? 

one  to  left? 

on  the  left? 

one  closest 7 

one  ahead7 

direct  left  or 
ahead  left7 

direct  right  or 
ahead  right? 

which 

doorway7 

which  room? 

which  wall7 

wheh 

OBJECT7 

1  see  multiple 
doorways 

1  see  multiple 
walls 

1  see  multiple 
OBJECT 

donl  see  door 
on  left 

don't  see  door 
on  right 

don’t  se« 
OBJECT 
DESCRIPTION 

unsure  wner*  to 

(topmowng 

torwaro 

unsura  where  to 
stop  going 
forward 

move  capability 
distance  or  object 

how  tar  forward 
go? 

how  far  move 
forward7 

how  far  back 
go? 

how  tar  back? 

how  tar 
DIRECTION7 

how  tar  go 
down  hallway 7 

how  far  move 
down  halfway? 

unsure  when  to 
stoptummo 

unsurs  whan 
should  stop 

turn  capability 
degrees  or  face 

how  tar  turn 

>  Chat Panel 

Chatroom:  /room2  |  i  connec 

Username:  wizard 

(17:59:04) 

system_message:  wizard 
has  joined  the  session 
(17:59:24)  wizard:  send 
image 

(17:59:40)  wizard:  move 
to  alley  Door  2 
(18:00:27)  wizard:  send 
image 


O  ChatPanel 

Chatroom:  /roomi  |  I  connec 

Username:  robot 

session 

(17:59:20)  robot:  done 
(1 7:59:26)  robot:  sent 
(17:59:37)  robot: 
processing. . . 
(18:00:04)  robot: 
moving. . . 

(18:00:23)  robot:  done 
(18:00:29)  robot:  sent 
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Results  from  Experiments  1  &  2  Ahl 


•  Experiment  1 :  People  use  a 
variety  of  language  when 
interacting  with  robots 


Landmark  vs.  Metric  Usage  in 
Instructions  to  Robot 


■  Landmark  ■  Metric 
*  p  <  0.05 


•  Experiment  2:  DM-Wizard  GUI 
improves  data  collection  by 
enabling  faster  data  collection 
(more  successful  interactions) 


#  Successful  Interactions  per 
Dialogue:  Experiment  1  vs.  2 


0  10  20  30  40  50 

#Sls  /  Trial 

**p<o.oi  14 
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Lifecycle  of  Human-Robot  Dialogue 
Development 


akl 


Phase  1  Exploratory  Data  Collection 
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End  Goal:  Autonomous 
Dialogue  Manager 


akl 


O  ®(100%)  TueAug  15  12:11  PM  # 


0  Time 


ROSTime:  B072.21  ROS  Elapsed:  100.74  Wall  Time:  M00.17  Wall  Elapsed:  116.45  Experimental 

Reset  31  fPs 


I  Time:  iterations 


n 


Move  Camera  L  J  Select 


HBtimate  X  2D  Nav  Coal 


F'>;l  : 


slukin@)NB21 003:  ~/svn_botlanguage/Users/slukin/py_scripts/de 


lukin@NB21003 : ~/svn_botlanguage/Users/slukin/py_scripts/demo$ 
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Summary 


AKL 


Research  Goal:  Studying  humans  having 
natural  dialogues  with  robots 

•  Contributes  to  natural  language  &  multimodal 
training  data  between  humans  and  robots 

•  Dialogue  offers  humans  a  way  to  supplement  a 
robot’s  knowledge  /  understanding  of  the  world 

•  Evaluating  ability  to  leverage  work  conducted  for 
virtual  humans  and  apply  to  robots 

•  Identifying  concepts  critical  to  human-robot 
dialogue 


Natural  language  control 


Proceed  forward 
until  you  reach 
the  target. 


Sgt.  Blackwell 

Virtual  Human 

Developed  at  ICT 


r 
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The 
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Thank  You! 
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Challenges 

How  is  language  used  in  dialogue  with 

technology  (vs.  human-human  dialogue)? 

What  will  robot  need  to  “know”  and  “see”  in 
order  to  have  common  ground  with 
humans? 

Army-unique  issues  include  operational 
environment  and  tempo,  vocabulary, 
non-structured  environment 


Current  State  of  the  Art 

Robot  has  limited  understanding  of  navigation 
and  manipulation  commands;  no  dialogue, 
only  simple  one-way  replies 


UNCLASSIFIED 
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Motivation 


Laboratory  for  Land  Forces 


UNCLASSIFIED 


Series  of  studies  planned  that  will  examine: 

»  Increased  automation  support  for  dialogue  management 
»  Multimodal  inputs  (e.g.,  head  nods,  eye  gaze) 

»  Human-robot  dialogue  in  more  complex  environments 
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Open  Campus  Initiative 


Akl 


Past:  Current  Defense 
Laboratory  Model 

Gates  and  high  walls  provide 
20th  century  security,  but  are 
barriers  to  21st  century 
innovation 


Defense  laboratories  relatively 
unchanged  since  inception 
(NRL1923) 


Present  &  Future:  Open  Campus  Initiative 

Reduction  in  barriers  to  facilitate  collaboration  with 
academia,  industry,  and  small  business 


Open  areas  for 
researchers  and 
access  to  existing 
facilities 


7f 


*5 


Hub  and 
Spoke 
Model 


Collaborator 

presence 

through 

EUL 


Career  path 
for  students 
and  scientists 


An  enhanced  defense  research  environment 
that  fosters  discovery  and  innovation  through 
collaboration  on  fundamental  research 


Collaboration 
between  ARL  and 
external  scientists 


UNCLASSIFIED 


The  Nation’s  Premier  Laboratory  for  Land  Forces 


Realizing  Our  Collective  Vision  by  2025: 
Leveraging  advances  in  Artificial 
Intelligence  and  Autonomy 
with  Human  Systems  in  Human- 
Machine  Symbiosis  to  Realize  Our 
Roadmap  to  the  Future  ... 
a  Cvber-Securitv  Workforce  Use  Case 


Dylan  Schmorrow,  PhD,  Chief  Scientist  & 
Executive  Vice  President 
Denise  Nicholson,  PhD,  VP  of  Intelligent 
Training  &  Director  of  “X” 


O 

SOARTECH 


Modeling  human  reasoning. 
Enhancing  human  performance. 


Human  Systems  in  i  f  I 
Emerging  Domainsri 
Autonomy,  Human  y# 
Augmentation  and  Cyber. 


O  soartech 


Outline 

•  Where  have  we  come  from 

-Early  concepts  in  Advanced  Embedded  Training 

•  Where  are  we  going 

-Personalized,  Life  Long  Learning 
-Sailor  2025  -  Ready  Relevant  Learning 

•  A  Use  Case  in  Cyber  Workforce  Training 

-  National  Initiative  for  Cybersecurity  Careers  and  Studies  (NICCS) 
Framework 

-Training  Learning  Architecture  in  conjunction  with  LVC  learning 
experiences 

•  Challenges  for  2025 

-Data  Data  Data 
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Human  Systems 


1  998  ACTD 

Advanced  Embedded  Training  System 


1  998  ACTD 

Advanced  Embedded  Training  System 


Student  Model:  Measured  Operator  Actions 


Online  Feedback 


Performance  Measurement 
Subsystem  - 
LM  ATL  &  TSD 


Diagnostic  Subsystem  - 
CHI  Systems,  Aptima  &  TSD 


Instructional  Subsystem  - 
Sonalysts  &  TSD 


Eye  Tracking 


Expected  Operator  Actions 
Generated  By  Expert  Models 


Comparator 

Automated 

Performance 

Assessment 

Engine 


Training  Mitigation  Director 

Operator  Action  Evaluations 
Focused  on  Scenario 
Key  and  Critical  Events 


Diagnosis  of  Knowledge 
And  Skill  Deficiencies 


Post-Exercise 
Debriefing 


Instructor  Hand-Held  Device 
_ ShipMATE _ 


O  soartech 


Sailor  2025  -  3  Key  Elements  of  RRL 


1.  Career-  Long 
Learning 
Continuum 

2.  Modern  delivery  at 
the  point  of  need 

3.  Integrated  Content 
Development 

-  Delivery 

methodologies 


Commander,  Ui.  Fleet  Forces  Command,  Executive  Agent  for  RRL 

Vision  and  Guidance 
for  Ready  Relevant 
Learning 

Improving  Sailor  Performance  and  Enhancing  Mission  Readiness 
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http://www.public.navy.mil/usff/rrl/Documents/PDFs/ 

rrl-vision-and-guidance-final.pdf 


Human  Systems 


O  soartech 
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USE  CASE  -  National  Initiative  for  Cybersecurity 
Careers  and  Studies  (NICCS) 


•Shortage  in  cyber  security 
workforce 

•Aid  in  pinpointing  what 
current  and  future 
professionals  need  to  know 
for  a  career  in  the  cyber 
workforce 

•Missing  link 


NICE 


Human  Systems 


[rf  *Ti 


O  soartech 


Training  and  Learning  Architecture  (TLA) 

•  Learner  Profiles 

-  Basic  information  regarding  the  user 

•  Content  Brokering 

-  Decision  making  on  what  type  of  content 
the  user  needs  to  cover  to  accomplish 
their  unique  goal 

•  Experience  Tracking 

-  Learner  profiles  updated  as  learner 
progresses  in  competency 

•  Competency  Network 

-  Library  of  course  content  to  be  pulled  by 
content  brokering  as  needed 


ffl,  NDIK 

mm*m mtmm  Human  Systems 


O  soartech 


Recommender  Ul 

Our  metaphor:  multiple  offerings  and 
rationales 


EIJVi.n 


Continue  Watching  for  Miss  A  L 


NETFUX 


METFIIX 


JESSICA 

JONES 


SWORDof 

DESTINY 


Roadmap 


Because  you  watched  How  to  Train  Your  Dragon 


in 


Social  Media 


HETUIX 


NETFUX 


ORANGE 


JESSICA 

JONES 


Album 


Playlist 


Alternate  /future  Uls  -> 


Human  Systems 


O  soartech 


Use  Case  with  TLA 


Career  Goals 


u  u 
O  J  O 


•V 


Recommendation: 
LVC  Exercise 
with  Al  Red  team 


KSA  #3  -  Computer 
Network  Defense  & 
Assessment  Tools 


Recommendation: 
Cyber  Mindset  Training 


KSA  #2  -  Adversary 
Tactics,  Techniques,  & 
Procedures 


Recommendation: 
CYSTINE  Simulation 


NICCS  Framework 


KSA  #1- Insider 
Threat 


Recommendation: 
SCITE  3D  Game 


Human  Systems 


O  soartech 
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KSA  #1:  Knowledge  of  and  experience  in  Insider  Threat 
Recommended  Activity:  3D  Insider  Threat  Game 


•Scenario  based  gaming 
environment  to 
experience  insider  threats 

•Occurs  within  an  office 
space  and  designed  to 
replicate  the  exploitation 
of  computer  systems  by 
employees  to  gain  access 
to  financial  information 
without  permission 


O 

SOARTECH 

Modeling  human  reasoning. 
Enhancing  human  performance, 


Human  Systems 


O  soartech 


KSA  #2:  Familiarization  w/  Common  Adversary 
Tactics,  Techniques,  and  Procedures 

Recommend:  Cyber  Security  Environment  (CYSTINE) 


Dynamic  training  scenario  that 
adapts  to  the  skill  of  the  trainee 

Cyber  defender  cognitive  agents, 
provide  dynamic,  cognitively 
realistic  adversaries 


Proficiency 
i  Progress 


CYSTINE 


Student 


Hyw*y  Assessment  a 

^  YO  YY*  Y  YYfc  YW  YYYYY**YYfcY*U  r  c  ^  n  a  | :  ^  J 

rersonanzea 

•  ,i*r Instruction  v 

^yc>cayway\\yy^yyy<ayc>^<a^y<a^yyyy^y^«,  i  4vi  i  i  y  i&< 


Plant  the  Flag 


In  thi*  media,  you  ore  t— >*d  vMh  qy  irqtniafrg  uw  Ury*i  maeftn#  r>  cadr  to(i*Mt  «  fey  * 
*pwnvd*«  no  qjoim.  In  ocdtKn.  yre  wtl  ba  a**«rO  to  ir  «««•  qjotf kto  along  iNr  *«y  «o  dam 
W3n«  -  utooa  actor  agaretam#c#irooirt30oc#yxif  BJCnrlad  nwJt  n  deoualf’caftcn 


Personalized 
Instruction  v 


TTc  IP&ddtKC  art  tre  taxet  rraenre  Is  '92.tGa.57J  Uirlt  \OJ  actors  tc  tfiiraohnc  takrc  action  acyani*  oitna'  nrt 
against  pert  22  on  tfa  targe*  tracers. 


I  W’u:  are  rwooan  TCP  ports  on  tft«  taxor?  rraanre.  ins  tax«  tiayire  c  ixcteo  ct  rto  ao»a*v»a  H  Aasrcst  '  92.  t CB. 
fit} 

&214 


iWUWimi  #  ^  A£><^AAttCA 

mmuw  Scenarios  Ny^BI 

LOYC^tA^CAYYYW^Y^Y^CtoYY^Y^YY^M 

A<A<AYCAYY^Y<A^C>YYY^YCA^Y^Y<AY*AYt^® 


»^Y^YYYY<AY<^^<AY< 
*■1  YY<AY<A^YY^YY^YY< 

|BLy^^a\\a^^a^^ 

M^yysayy^waywayw 


As  the  student  gets  better, 
the  scenarios  gets  harder 


Human  Systems 


O  soartech 


ln-attentional  Blindness  Exercise: 


KSA  #3:  Knowledge  of  Computer  Network  Defense 
Recommendation  -  Cyber  Mindset  Training 

Cyber  basics:  Prepare  non-cyber  personnel  to  participate 
in  largescale  synthetic  cyber  training 

*  Interactive  instruction  on  how  to  minimize  cognitive 
bias  interference 

*  Challenge  assumptions  about 
immunity  to,  e.g., 

•  Attentional  tunneling 

•  ln-attentional  blindness 

•  Confirmation  bias 

Game-based  event  recognition 
practice 

*  Develop  perceptual  sensitivity 

*  Gain  appreciation  for  importance  of 
maintaining  system  awareness 


Find  the  change  between  the  images 


a 


-w  r 1  r 


/*  H  ! 1 
A  LI 


etween  the  images 


\ 


/ 
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KSA  #3:  Knowledge  of  Computer  Network 
Defense  and  Vulnerability  Assessment  Tools 
Recommend  -  LVC  Exercise  with  Al  Red  Team 


OPERATION 

BLENDED 

WARRIOR 


ft&dmes  Tfmugfi  £VC 


Simulated  Cognitive  Cyber 

Red-team  Attack  Model  SC2RAM 

allows  training  exercises  to  be  implemented  on  a 
scale  that  adaptable  to  the  emerging  professionals 


Human  Systems 


O  soartech 
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Future  Challenges  -  Data,  Data,  Data 
Data 

•About  the  activities/training 
-Learner  progress  thru  the  activity 
•About  existing  content  -  reuse 

Data 

•Competencies 

-  Personal  Qualification  Standards 
•  Learner  models 

-thru  the  activity  not  just  complete 

Data 

•Security  &  IA 

-  Readiness,  Personal  data  protection 


Competencies 


— 


% 

* 

# 


Human  Systems 


O  soartech 
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QUESTIONS  and  DISCUSSION 


Human  Systems  in  f 
Emerging  Domains:  1 
Autonomy,  Human 
Augmentation  and  Cyber. 


Dylan  Schmorrow 

dvlan.schmorrow(a)soartech.com 

Denise  Nicholson 

demise.  Nicholson  (5)  soartech.  com 


Human  Systems 


Integrity  *  Service  *  Excellence 


Human-System  Metrics 
Applied  to  Optimize  AF 
Warfighter  Capability 

13  March  2018 


NDIA  Human  Systems  Conference 
Ms.  Sarah  Orr 

Human  Systems  Integration  Directorate 
711th  Human  Performance  Wing 


Distribution  A.  Approved  for  public  release:  distribution  unlimited.  PA  Case  Number:  88ABW-2018-0509.  Date  cleared:  05  Feb  2018 


Overview 


711  HPW/HP  -  AF  HSI  Analysts 
o  Providing  support  across  AF 
o  Other/joint  efforts 
HSI  -related  requirement  activities 
o  During  early  system  concept  development 
o  Throughout  the  product  lifecycle 
o  Human-System  Requirements 
s  Methodology 
s  Measures  and  Verification 
•/  Metrics 

Example:  application  of  HSI-related 
requirements  in  emerging  domains: 
autonomy,  human  augmentation,  and  cyber 
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Optimize  warfighter  capability  through  a 
human-centric  approach  to  system 
development,  acquisition,  and  sustainment 
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Optimize  total  system  performance 

...the  relative  and  combined  performance 
...human  +  software  +  hardware 
...trade  offs 

Optimize  total  life  cycle  costs 

...projecting  sustainment  costs 
over  30-70  years  (adds  up)! 

Enable  the  users  to  effectively  complete  the  mission 

HSI  analysts  work  to  ultimately  reduce  overall  program  risk 
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JCIDS: 
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System  level  HSI-related  requirements: 

•  Are  performance  based.  "A  performance-based  requirement  states 
'how  well'  a  function  must  be  performed.  The  performance  is  usually 
indicated  by  a  measurable  value." 

Source:  USAF  HSI  in  the  SRD  Guidebook,  SURVIAC,  2012 

•  Are  accompanied  by  verification  methodology 

•  Are  often  accompanied  by  SOW  tasking  for  associated  analyses, 
modeling  and  sim,  and  formative  evaluations 
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Measures  versus  Metrics: 

•  Measure  is  used  here  for  more  concrete  attributes 

•  Metric  is  higher-level;  a  standard  of  measurement 

•  "Metrics  can  be  used  to  benchmark  and  measure  performance 
against.  Metrics  are  measures  collected  overtime  for  the  purpose 
of  seeing  trends  and  forecasting  program  progress  to  plan." 

-  DAG  Chapter  3 
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Human-System  Requirements 


Basic  Requirement  Statement: 


<System  D>  shall  provide  <F>  feedback  within  <E>  seconds. 


Measure:  a  clear  and 
measureable  pass  / 
fail  criterion 


Source:  USAF  HSI  in  the  SRD  Guidebook,  SURVIAC,  2012 
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HSI  Involvement 


Levels  of  HSI  Requirements  and  Verification 


rv 


JCIDS  CDD 
Level 


KPP-Survivability 


Government 

SRD  Level 

Functional 

o 

4-> 

U 

Baseline 

CO 

i- 

+■> 

System  Spec 

(SRR) 

o 

u 

Level 

Emergency  Egress:  Passengers 
and  crew  shall  be  able  to  exit  in 
less  than  90  sec. 

Verify  by  Demonstration 


Emergency  Egress:  Passengers 
and  crew  shall  be  able  to  exit  in 
less  than  90  sec. 

Verify  by  Demonstration 


SRD  is  AF  equivalent  of 
System  Performance  Specification 
System  Level  (placed  on  contract) 


Z 


Allocated 

Baseline 

(PDR) 


Sub-System 
Spec  Level 


Product 

Baseline 

(CDR) 


Design  Level 


The  Emergency  Door  shall  be 
able  to  be  removed  by  <X 
number  of>  passengers  or 
crew  in  less  than  5  seconds. 
Verify  by  Demonstration 


Exit  door  sizes  must  be  A"  x  B" 
Verify  by  Inspection 


In  the  event  of  power  loss,  the 
system  shall  provide  a  visual 
means  to  assist  with  emergency 
egress  of  passengers  and  crew 
for  <Y  amount  of  time>. 
Verify  by  Demonstration 


Emergency  floor  lighting  shall 
consist  of  product  XXX-YYY. 
Verify  by  Inspection 


JCIDS  =  Joint  Capabilities  Integration  and  Development  System;  SRR  =  System  Requirement  Review;  PDR  =  Preliminary  Design  Review;  CDR  =  Critical  Design  Review. 
Source:  USAF  HSI  in  the  SRD  Guidebook,  SURVIAC,  2012 
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Methodology 
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Target  Audience  Description 


Operators  Maintainers  Support  Team  Other 


e.g..  Trainers 


e.g.,  Passengers, 
Other  FoS  Users, 
Bystanders 
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Part  of  the  overall 
systems  engineering 
process  -  a 
team  effort 


Mission  Analysis 


I 


Function  Analysis 


Decompose  mission 
capabilities  into 
functions  and  tasks 


I 


Task  Analysis 


as  part  of  the  overall 
system 

Human  capabilities 
and  limitations 
factored  into  the 
design  EARLY 

MIL-STD-46855A 
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Commercial  off  the  shelf 
or  transitioning 
technologies 


Mission  Analysis 


Function  Analysis 


I 


Task  Analysis 


Factors  into  Systems  Engineering  top  down 
requirements  decomposition  process 


Empowering  the  human  to  excel 
in  tasks  that  they  do  better  than 
machines 

Designing  hardware  and 
software  to  assist  humans  when 
machines  can  do  tasks  better 
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Notional  example: 

Security  Forces  need  assistance  patrolling  military  bases  and  apprehending  threats. 

The  following  capabilities  will  be  acquired  to 
augment  the  existing  forces: 

'Security  Forces  Bot  (SFB)' 

-  Autonomous,  mobile  patrolling  of  perimeters  of 
military  bases 

-  Sense  and  detain  airborne  intruders  (UAVs) 

-  Sense  and  detain  intruders  on  foot 

-  Work  as  a  team  with  other  SFBs  and  also  with 
[human]  security  forces 
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Example 


Total  system  performance 

...the  relative  and  combined  performance  of  humans  and  systems 


Empowering  the  human 
to  excel  in  tasks  that  they 
do  better  than  machines 


Designing  hardware  and 
software  to  assist  humans 
when  machines  can  do 
tasks  better 
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Example 


5ABW-201 8-0509.  Date  cleared:  05  Feb  2018 


Example 


Methodology:  Begin  with  the  Target  Audience  Description 


Operators 
Maintainers 
Support  Personnel 

Other 
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Example 


Net-centric  Ops 


Intruders  on  Foot 


Operational  View  -  With  Humans 


Military  Satellites 


Security 

Forces 


Other 


Threats 


Air  Traffic  Control 


Command  &  Control 
Ground-Based 


Friendly  "intruders 
children,  adults, 
pets,  wildlife 
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w 


Example 


Consider  the  following  areas  of  focus 


Cyber 


Augmentation 


Autonomy 
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Command  and 
Control  (C2) 
Operator 


Distribution  A.  Approved  for  public  release:  distribution  unlimited.  PA  Case  Number:  88ABW-201 8-0509.  Date 


Human-  System  requirements  might 
address: 

•  Alerting/notification  of  cyber  threats 

O  Software-related 
O  Hardware-related 
O  Overall  system 
O  Individual  SFB  cyber  issues 

•  Decision-aiding 

O  Responding  to  cyber  threats 
O  Understanding  the  impact/scope 
of  cyber  threats 

•  Ability  to  easily  update  system  to 
respond  to  new  emerging  threats 


The  C2  Interface  shall  provide  recommended  courses  of  action  to  avert  <type  A>  threat 
to  the  C2  Operator  within  <D>  seconds  of  detection.  Verify  by  demonstration. 


1^1  l\J  I  IU  VV  V^IIIV^I  III  II  II  uuu 


Human-  System  requirements  might 


Alerting/notification  of  cyber  threats 
O  Software-related 
O  Hardware-related 
O  Overall  system 
O  Individual  SFB  cyber  issues 
Decision-aiding 

O  Responding  to  cyber  threats 
O  Understanding  the  impact/scope 


C2  Interface 
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SF 

Interface 


Example 


Security  Forces  (SF) 
Operator 


Human-  System  requirements 
might  address: 

•  Teaming 

•  Communication 

•  Decision-aiding 

•  Override  of  automation 

•  Training 

•  Socialization/trust 

•  Safety 

•  Survivability 

•  Maintainability 
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SF 

Interface 


Augmentation 


1 


Human-  System  requirements 
might  address: 

•  Teaming 

•  Communication 

•  Decision-aiding 

•  Override  of  automation 

•  Training 

•  Socialization/trust 

•  Safety 


The  SF  Interface  shall  allow  the  trained  SF  Operator  to  override  autonomous  operation 
of  a  single  SFB  to  manual  control  within  <C>  seconds.  Verify  by  demonstration. 
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Human-Systems  requirements  might  address: 

•  System  protected  from  unauthorized  users 

•  Easy  changes  to  software  modules  (by  authorized 
users)  -  command,  control,  and  teaming  "rules" 

•  Communication 

•  Decision-aiding 

•  Safety 

•  IFF/Survivability  -  Friendly  "intruders" 

children,  adults,  pets,  wildlife 

•  Maintainability- diagnostics 

•  Sustainment-  #  of  people  required  to  support  and 
sustain  over  the  lifetime  of  the  system 

•  Training-  understanding  evolution  of  changes  to  SFB 


Autonomy 
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Human-Systems  requirements  might  address: 

•  System  protected  from  unauthorized  users 
:asy  changes  to  software  modules  (by  authorizei 
users)  -  command,  control,  and  teaming  "rules" 
Communication 
Decision-aiding 
Safety 


IFF/Survivability-  Friendly  "intruders" 
children,  adults,  pets,  wildlife 
Maintainability-  diagnostics 
Sustainment-  #  of  people  required  to  support  and 


Autonomy 


The  SFB  software  shall  be  capable  of  field  updates  for  <teaming  rules>  performed  by 
trained  SF  Maintainers  within  <D>  minutes.  Verify  by  demonstration. 


Distribution  A.  Approved  for  public  release:  distribution  unlimited.  PA  Case  Number:  88ABW-201 8-0509.  Date  cleared:  05  Feb  2018 


711  HPW/HP  -  AF  HSI  Analysts 
-  Postured  to  optimize  warfighter  capability  within 

current  and  emerging  technologies 

HSI  Lab 

•  Anthropometries 

•  Task  Analysis 

•  Usability 

•  Exoskeleton  augmentation 
Expanding  talented  HSI  analyst  workforce 

•  Cyber  SMEs 

•  Engineers 

•  Acquisition  professionals 

•  Doctors 

•  Physiologists 


Distribution  A.  Approved  for  public  release:  distribution  unlimited.  PA  Case  Number:  88ABW-201 8-0509.  Date  cleared:  05  Feb  2018 


26 


HP  workflow: 

711  HPW.HPWorkflow@us.af.mil 
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TM  9-XXX-XXX-10 


OPERATOR'S  MANUAL 
BASELINE  AND  PMCS 


HUMAN,  COMBAT,  MULTI-DOMAIN 


HEADQUARTERS,  DEPARTMENT  OF  THE  ARMY 

DATE 


Human -10  PMCS 
Baseline  -  Preventive  Maintence 

Checks  and  Services 


Physical  -  Cognitive  -  Belief 


1.  What  Should  We  Measure? 

2.  What  Can  We  Measure? 

3.  How  Do  We  Measure?  (Tools) 

4.  How  Well  Does  the  Tools  Measure? 

5.  What  Tools  Do  We  Need  to  Measure 
Better? 


ARDEC 


Optimizing  Future  Soldier  Systems 


through  the  Incorporation  of 


a  System  Domain  using  the 
Systems  Modeling  Language 

Sean  F.  Pham 
Shauna  M.  Dorsey,  Ph.D. 

NDIA  Human  Systems  Conference,  13  March  2018 


UNCLASSIFIED//DISTRIBUTION  A.  APPROVED  FOR  PUBLIC  RELEASE:  DISTRIBUTION  UNLIMITED. 


UNCLASSIFIED//DISTRIBUTION  A 


Problem:  The  U.S.  Army  has  historically  focused  on  the  development  and  optimization  of  Soldier 
equipment,  leading  to  integration  challenges  between  Soldiers  and  their  equipment. 


Best  Helmet 


Best  Small  Arms 


Best  Uniform 


Best  Boots 


A\  /  i 

Best  Load  Carriage 


Best  Body  Armor 


Best: 

•  Unmanned  aerial  vehicles 

•  Operational  rations 

•  Organization/leadership 

•  Quality  of  life  standards,  etc. 


It’s  not  just  about  Soldier  equipment.  We  must  also  understand  and  predict  the  performance  of  the 
full  system,  inclusive  of  the  Soldier,  his/her  equipment,  and  the  tasks  he/she  must  perform. 


13  March  2018 
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UNCLASSIFIED//DISTRIBUTION  A 


\  HUMAN  SYSTEM 


Objectives:  Create  a  principle-based  Soldier  architecture  and  framework  to  enable  a  system 
level  tradeoff  analysis  of  the  Soldier  as  a  System  (SaaS)  domain. 


Create  the  foundation  for  design  parameters 

for  the  next  generation  of  Soldier  systems  and 
subsystems,  which  considers  the  complete 
Soldier  as  a  System  with  the  full  complement 
of  equipment,  the  human  performance 
capabilities,  and  the  mission  tasks. 


HUMAN 

VIEWPOINT 


COMPATIBILITY 


COMPETENCY 


Anticipated  Outcomes: 

•  Increased  efficiencies  and  optimized 
performance  of  the  Soldier  as  a  System. 

•  Enterprise  approach  across  Soldier-Small  Unit 
Science  and  Technology  (S&T)  efforts,  combat 
developers,  and  acquisition  communities. 


\  SOLDIER ' 
CAPABILITY 


INTEGRATION  CAI 

'YsoldierV'" 

SYSTEM 


UTILITY 


13  March  2018 
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UNCLASSIFIED//DISTRIBUTION  A 


Purpose:  Utilize  Systems  Engineering  tools  and  processes  to  allow  stakeholders  across  the 
Soldier  Enterprise  to  manage  the  overwhelming  complexity  of  the  Soldier  as  a  System  domain. 


Soldier 


Operating 

Platform 


Utility 
3)  Durability 


Equipment 


SET  Framework 


MOEs 

A.  Suitability 

B.  Effectiveness 

C.  Survivability 


Soldier  System  Engineering  Architecture 
(SSEA)  is  integrating  these  tools  and 
processes  for  the  Soldier  Enterprise. 
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US.  ARMY 


Soldier  System  Engineering  Architecture 


Soldier  as  a  System  (SaaS)  Reference  Model: 


-  Characterizes  the  Soldier  as  a  System  domain  in  terms  of  the 
human  dimension,  the  materiel  solutions,  and  the  operational 
environment  (i.e.,  the  Soldier,  Equipment,  Task  [SET] 
framework). 


-  Formalizes  the  definition  of  the  SaaS  domain. 


Elements  of  the  Soldier,  Equipment,  and  Task,  along  with 
their  interactions  and  interrelationships. 


System  Modeling  Language  (SvsML): 


-  Captures  the  system  model  and  defines  the  boundaries  of 

the  system  space. 


Enables  decomposition  of  the  SaaS  domain  and  establishes 
a  common  vocabulary. 


-  Provides  a  common  underpinning  for  SSEA,  allowing 
stakeholders  to  further  understand  their  piece  of  the  SaaS 
domain  and  its  impact  points  over  the  full  system  space. 
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Purpose  of  the  Model  Structure: 

-  Define  the  domain/system  space 
(SaaS)  and  boundaries. 

-  Serve  as  a  central  hub  for  the  defined 
SaaS  components  and  relationships. 

•  Comprised  of  the  Soldier  system 
within  an  operational  context. 

•  Displays  any  interrelationships 
between  the  primary  model 
components. 
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Scenario:  Soldier  engaging  an  enemy  target. 
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US.  ARMY 


Soldier,  Equipment,  and  Task  Segments 


^Srdi ^ 


Purpose:  Define  the  elements  and  relationships  contained  within  Soldier,  Equipment,  and  Task 
(SET)  segments  of  the  Soldier  as  a  System  (SaaS)  model. 


SET  Framework 


Operating 

Platform 


Soldie 


Soldier  Segment 


ical.  Cognitive,  and  Social 

Parameters 


Orient 


Terrain 


Material 

Development 


Defend 


Features 


Mitigation,  Sense,  Affect,  &  Communicate 


MOPs 


MOEs 


1)  Applicability 

2)  Utility 

3)  Durability 


A.  Suitability 

B.  Effectiveness 

C.  Survivability 
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Purpose:  Define  the  elements  and  relationships  within  the  human  dimension,  which  includes 
cognitive,  physical,  social,  and  emotional  parameters  to  further  characterize  the  Soldier. 


Soldie 


Terrain 


Material 

Development 


Defend 


Features 


Mitigation,  Sense,  Affect,  &  Communicate 


MOPs 


SET  Framework 


Operating 

Platform 


Soldier  Segment 


cal,  Cognitive,  and  Social 

Parameters 


Observe 


Decide 


Orient 


Act 


MOEs 


Ip  Applicability 

2)  Utility 

3)  Durability 


A. 

Suitability 

B. 

Effectiveness 

C 

Survivability 
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Soldier  as  a  System:  Soldier  Segment  of  the  Model 


Soldier  Block  Definition  Diagram 


m  -E 


E 


«block* 

«Characteristic» 

«SoWier» 

Cardiovascular  System 


Specific  Classifier  = 
^Unhealthy 
^Susceptible 
^Healthy 


:  Blood  Vessels 
Heart 


Physiological  Load 


Four  Main  Components: 

1 .  Anthropometry  -  Physical  structures  of  the  human 

2.  Physiology  -  Internal  regulatory  systems  of  the  human 
Behavior-  Voluntary  (i.e.,  cognitively  founded)  and  reflexive  (i.e., 
“hard-wired”)  behaviors 

Intelligence  -  Fluid  (i.e.,  creativity  and  learning),  crystalized  (i.e., 
prior  skills  and  knowledge),  social,  and  emotional  intelligence 


3. 


4. 


Component  Classifiers: 

•  Size  and  shape 

•  Health  state 

•  Response 

•  Creativity  and  learning 

•  Education  and  experiences 

•  Communication  style 

•  Emotions 


Ports  /  Interactions  (examples): 

•  Shoulder/  Support,  Stabilize 

•  Hand  /  Support,  Secure 

•  Finger/  Control  Magnitude,  Actuate 

•  Eye  /  Signal  Sense 

•  Body  /  Support,  Secure,  Attach 
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Purpose:  Define  the  elements  and  relationships  within  the  material  development  dimension, 
including  the  type,  form,  and  function  of  the  equipment  and  how  it  relates  back  to  its  requirements. 


Soldier  Segment 


Physical,  Cognitive,  and  Social 

/ _ Parameters 


Observe 


Decide 


Orient 


Terrain 


Material 

Development 


Delend  A  Features 


Mitigation,  Sense,  Affect,  &  Communicate 


MOPs 


SET  Framework 


Operating 

Platform 


MOEs 


" Applicability^ 

2)  Utility 

3)  Durability 


A. 

Suitability 

B. 

Effectiveness 

C 

Survivability 

_ 
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Soldier  as  a  System:  Equipment  Segment  of  the  Model 


Equipment  Block  Definition  Diagram 


((Characteristic]!) 

((Equipments^ 

«block» 

Equipment  Form 
Specific  Classifier  = 

rd  Body-worn 
Q  Carried 
BConsumed 
HHead-worn 
^Operated 
L 


Two  Components: 

•  Equipment  Form  -  Integrated  weapon  system, 
clothing,  and  individual  equipment 

•  Equipment  Function  -  Combat  casualty  care, 
mobility,  protection,  mission  command, 
lethality,  logistics  support 


Component  Classifiers: 

•  Forms  of  Equipment 

•  Body-worn 

•  Carried 

•  Consumed 

•  Head-worn 

•  Operated 


Ports  /  Interactions  (examples): 

•  Buttstock/ Support,  Secure 

•  Improved  Outer  Tactical  Vest  /  Support,  Stop,  Protect 

•  Rucksack/ Provision,  Store,  Hold 

•  Close  Combat  Optic  /  Channel,  Import,  Allow 

•  Eye  Protection  /  Control  Magnitude,  Regulate 
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Purpose:  Define  the  elements  and  relationships  that  the  Soldier  will  encounter  within  a  specific 
operational  environment.  This  focuses  primarily  on  doctrinal  mission  elements  and  parameters. 


Terrain 


Features 


Individuals 


Mitigation,  Sense,  Affect,  &  Communicate 


MOPs 


SET  Framework 


Operating 

Platform 


Soldier  Segment 


Physical,  Cognitive,  and  Social 

Parameters 


Obsei  i/e 


Decide 


Orient 


c 

%  o^x 

>  a  R 


Material 

Development 


2)  Utility 

3)  Durability 


MOEs 


A. 

Suitability 

B. 

Effectiveness 

C 

Survivability 
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Soldier  as  a  System:  Task 


-  b 

e  Mission 

■mSLo 

TN*  tfiftram  ■»«  prtmry 

1 

1 

1 

PtiywK*  {nwonnwrt 


a«wco*i 

QCENTCOM 

□EUCOM 

□kortwcom 


gSOUTHCam 


Tasks 


UNCLASSIFIED//DISTRIBUTION  A 

MENT  OF  THE  MODEL 


t 


Task  Block  Definition  Diagram 


9p*CM<  ClUXt*! « 
OMIHtaryCanMItt 
C^SoCtM  C  Jf!  jMift 


<ilrrtj>»uajr«  caoM 


oDoctnfinMT»<iic» 

<■>  Support  am)  RMMonalupa 
<■*  Oudrwuc  T*cnmcji  Data 


SlMkCiasttw 
a^Ropul*  Tukl 
QkrtgutaiTnts 


QMuPuckirt  Pto«l« 


« ■ ask» 

((Required  Capability  * 
Warfighting  Functions 


Specific  Classifier^ 

■^Mission  Command 
■^Sustainment 

cjManeu ,  er  Support  and  Protection 
■^Engagement 
■^Movement  and  Maneuver 
■Q  Fires 
■^Intelligence 


Four  Components: 

1.  Physical  Environment  -  Terrain,  climate,  structures  (man¬ 
made  or  natural),  and  regional  areas 

2.  Relevant  Actors  -  Organizations  and  people 

3.  Operational  Behavior  and  Activity  -  Coalition,  host  nation, 
and  enemy  activities,  along  with  civil  considerations 
Unified  Land  Operations  -  Characterizes  decisive  actions, 
warfighting  functions,  and  doctrinal  tasks 


4 
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Component  Classifiers: 

•  Types  of: 

•  Terrain  and  climate 

•  Physical  structures  and  areas 

•  Groups  and  personnel 

•  Operational  variables  (HAMO) 

•  Operational  activities 

•  Threats  and  actions 

•  Tasks  and  functions 
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Purpose:  Standardize  methods  and  elements  to  depict  the  relationships  between  the  Soldier, 
Equipment,  and  Task  segments  of  the  SaaS  model. 

Interaction:  Soldier  Shoulder  to  Rifle  Buttstock  in  an  active  “engagement”  position. 


Soldier 

Shoulder: 


Human  Force 

Surface  for  Physical  Contact 

- N 


Secure^ 


Mechanical  Force 
Surface  for  Physical  Contact 
Tactile  Signal 


A 


Rifle 

Buttstock: 


Stabiliz 


Otto  K  and  Wood  K.  Product  Design:  Techniques  in  Reverse 
Engineering  and  New  Product  Development,  1st  Ed.  2000. 
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Conclusions 


•  SysML  SaaS  model  captures  a  system  of  interest, 
which  include  elements  related  to  the  Soldier, 
equipment,  and  task  capabilities. 


•  Human  systems  integration  aspects  are  captured  to 
further  depict  the  relationships  between  the  Soldier 
and  their  equipment  in  an  operational  context. 


•  SSEA  SysML  models  can  be  used  as  a  tool  to 
improve  decision  making  through  a  better 
understanding  of  Soldier-equipment  interactions, 
leading  to  the  optimization  of  future  Soldier  systems. 
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U.S.  Army  Armaments  Research,  Development,  and 

Engineering  Center  (ARDEC): 

•  Shauna  Dorsey 

•  Frank  Torres 

•  Dana  Perriello 

•  David  Chau 

U.S.  Army  Natick  Soldier  Research,  Development, 

and  Engineering  Center  (NSRDEC): 

•  David  Krasnecky  (STO  Manager) 

•  Michael  Curry  (Draper) 

•  Axel  Rodriguez 

•  Joseph  Patterson 

•  Roger  Schleper 

•  John  Turkovich  (Draper) 

•  Jeff  Cipolloni  (Draper) 


Other  Government  Agencies: 

•  U.S.  Army  Research  Lab  (ARL) 

•  U.S.  Army  Aberdeen  Test  Center  (ATC) 

•  U.S.  Army  Communications-Electronics 
Research,  Development,  and  Engineering 
Center  (CERDEC) 

•  U.S.  Army  Medical  Research  and  Material 
Command  (MRMC) 


U.S.  ARMY 

P  RDECOM 


7  Always  a  Step  Ahead 

ARDEC 

ARMAMENTS 
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Thank  You 


Questions? 
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Human  Systems  in  Emerging  Domains: 
Autonomy,  Human  Augmentation, 

and  Cyber 


1 


V 


r.  Paul  Zablocky 
Chief  Scientist  (Acting) 

March  14,  2018 
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Lessons  from  History 


.  .plan,  foster  and  encourage  scientific  research 
in  recognition  of  its  paramount  importance  as 
related  to  the  maintenance  of  future  naval  power, 
and  the  preservation  of  national  security...” 
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The  Naval  Research  Model  c.  1947 


“It  is  of  the  utmost  importance  to  our 
national  security  that  the  Navy  prosecute 
a  vigorous  and  well-rounded  program 
of  research  and  development. . .  ” 

-  Secretary  of  the  Navy  James  V.  Forrestal,  January  1947 


Government 


Defense  Industry 
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Cold  War  Success  Stories 


Ballistic  Missile  Submarine 


Government 


Academia 


Defense  industry 
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Today’s  Challenge 
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U.S.  Naval  Superiority  is  NOT  Guaranteed 


Naval 

Power 


Networked 

Platforms 


/ 

Current 

Fleet/ 

Force 


More  / 
Capable 
****  Platforms 


More  ** 
Platforms 


and... 

Need  Faster 
than  Current 
Pace  Allows! 


Renewed  Urgency  in  R&D  is  Needed  to  Win 
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The  Naval  Research  Ecosystem  c.  2018 


Global  Commercial  Market  Driven 


YORK, 


iioSli-lfonT  im 


Iprooklun 
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,  no  ii; 
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>iemens  Bi 


SAMSUNG 
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MW  gEBTY^i-- 
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Virginia 


amazon 
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Naval  Research  Response 


FRAMEWORK  PRIORITIES 


Augmented 

Warfighter 


Integrated  & 
Distributed  Forces 


Operational 

Endurance 


Scalable 

Lethality 


Sensing  & 
Sense-Making 


“...the  tempo  of  modern  war  has  reached  the  point  where  this 
Nation  will  probably  never  again  have  an  opportunity  to  arm 
itself  successfully  after  the  start  of  hostilities....  ”  -  Forrestal 


Distribution  Statement  A:  Approved  for  public  release  8 


Research  &  Development  Priorities 


Augmented 

Warfighter 


Integrated  & 
Distributed  Forces 


Scalable 

Lethality 


Sensing  & 
Sense-Making 


Five  Framework  Priorities 
that  are  Strategic  and 
Warfighter-Focused . . . 


Mission  Capable, 
Persistent  & 
Survivable 
Sea  Platforms 

Warfighter 

Supremacy 
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Operational 

Endurance 


Amphibious 

Expeditionary  Maneuver 


Information,  Cyber  & 
Spectrum  Superiority 

Undersea 
Battlespace  & 
Maritime  Domain 
Access 


...Translates  to  Six 
Technology-Focused 
Integrated  Research  Portfolios 


Aviation,  Force  Projection 
&  Integrated  Defense 


Naval  Research  Enterprise  Addendum  to  the 

NAVAL  RESEARCH  AND  DEVELOPMENT 
FRAMEWORK 


NRE  Framework 
Addendum 


55  Enduring  Research  Responsibilities 


EXPEDITIONARY  FIRES  AND  LETHALITY 

EXPEDITIONARY  C4ISR 

HYBRID  THREAT  DEFEAT 

HUMAN  PERFORMANCE  AND  PROTECTION 

AMPHIBIOUS  MOBILITY 

LOGISTICS,  SUSTAINMENT  AND 

MAINTENANCE 

EXPEDITIONARY  POWER  AND  ENERGY 
LIGHTENING  THE  LOAD 
ACCELERATED  LEARNING/DECISION¬ 
MAKING 

INFORMATION  ENVIRONMENT 
OPERATIONS 

DIRECTED  ENERGY  (DE)  &  COUNTER  DE 

AERODYNAMICS 

FLIGHT  DYNAMICS  &  CONTROL 

PROPULSION 

STRUCTURES  AND  MATERIALS 
ENERGETIC  MATERIALS 
HYPERSONICS 
AUTONOMY 


•  ADVANCED  RF  ELECTRONICS  & 
MATERIALS 

•  COMMUNICATIONS  AND  NETWORKING 


•  COMPUTATIONAL  METHODS  FOR 
DECISION  MAKING 

•  DATA  SCIENCE  AND  ANALYTICS 

•  ELECTRONIC  WARFARE 


•  SENSORS  AND  SENSOR  PROCESSING 

•  MACHINE  LEARNING,  REASONING  AND 
INTELLIGENCE 

•  RESOURCE  OPTIMIZATION 

•  PRECISION  NAVIGATION  &  TIMEKEEPING 


•  UNDERSEA  MEDICINE 


•  BIOLOGICAL  SCIENCES 

•  BIOROBOTICS 


•  CAPABLE  MANPOWER 

•  COMMAND  DECISION  MAKING 

•  FORCE  HEALTH  PROTECTION 

•  HUMAN-ROBOT  INTERACTION 

•  NOISE-INDUCED  HEARING  LOSS 

•  TRAINING  AND  SIMULATION 


ARCTIC  AND  GLOBAL  PREDICTION 
LITTORAL  GEOSCIENCES  AND  OPTICS 
MARINE  MAMMALS  AND  BIOLOGY 
MARINE  METEOROLOGY 
MARITIME  SENSING 
OCEAN  ACOUSTICS 

OCEAN  ENGINEERING  &  MARINE  SYSTEMS 
PHYSICAL  OCEANOGRAPHY 
RESEARCH  FACILITIES 
SPACE  ENVIRONMENT 
UNDERSEA  SIGNAL  PROCESSING 
NAVAL  ENGINEERING 
ADVANCED  NAVAL  POWER  SYSTEMS 
ADVANCED  SURVIVABLE  SEA 
PLATFORMS 

UNMANNED  SEA  PLATFORMS,  AUTONOMY 
AND  POWER 

ADVANCED  NAVAL  MATERIALS 
UNDERSEA  WEAPONS,  COUNTER¬ 
WEAPONS  AND  ENERGETICS 
SEA  PLATFORM  ENVIRONMENTAL 
QUALITY 

CORROSION  CONTROL  10 
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Getting  to  Capability 


The  ONR  Portfolio  is  Broad  Yet  Singularly  Focused 
on  Delivering  Continued  Naval  Superiority 


Efforts  that  have  moved  into  the  development  and 
integration  of  hardware  for  field  experiments  and  tests. 


$879M 


Advanced 

Technology 

Development 


Evaluation  of  integrated  technologies  in  as  realistic 
an  operating  environment  as  possible  to  assess  the 
performance  or  cost  reduction  potential. 


$956M 


Applied  Research  to  determine  the  means  by  which 
Research  a  recognized  and  specific  need  may  be  met. 


$547M 


Basic  Research  without  specific  applications  toward 

Research  processes  or  products  in  mind. 
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FY17  Funding 


The  Portfolio  Investment 
Relative  to  FY17  Navy  Budget 


Advanced  Technology 
Development 
$879M 


Basic  Research 
$547M 


Applied  Research 


$956M 


Demonstration 
and  Validation 
$37M 


FY17  DoN  BUDGET 
$160.9B 


Nava.  Researeh  Enterprise 


A1,otted$2.42Bo,fV17RDT&E  Budge, 
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The  Naval  Research  Enterprise 


u.s.  NAVAL 
LresearchJ 

LABORATORY 

NRL 


ONR  Global 


4,000+  People 
23  Locations 
$2.1  B /year 
> 1,000  Partners 
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ONR  Headquarters 
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Naval  Research  Laboratory 


•  The  Navy’s  Corporate  Laboratory 

•  World  Class  Research  Team 

•  Basic  and  Applied  Research  and  Advanced  Technology 
Development  for  Anticipated  Navy  and  Marine  Corps  Needs 

NRL  Products  &  Prototypes 


Delivering 

Warfighter 

Advantage 


Directly  Funded  Work  Integrated  Research  Portfolio 
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ONR  Global 


Washington,  D.C.  Region  /  ONR  HQ 

ONR  Global  Arlington 

Executive  Officer 


International  S&T  Engagement  Advisors 


FCC/C10F  kkk  \ 
N81  ★★★ 


San  Diego 

C3F  kkk****. 
AIRFOR  kkP* 
SURFOR  ★  ★★ 

I  MEF  ★  ★★ 
UWDC  ^ 
SMWDC^r 


p 

Hawaii** 

PACOM 

PACFLT  ★★★★ 
MARFORPAC  ★  ★★ 


Camp  LeJeune 

II  MEF  ★  ★★ 


ONR  Global  Santiago 

Science  Director 


Science  Advisors 

k  Joint  Command 
^  Navy  Command 
k  Marine  Corps  Command 


ONR  Global  Headguarters  London 

Commanding  Officer 
Technical  Director 
Science  Directors 


Norfolk 

FLTFOR  ★  ★★★ 
SUBFOR  ★  ★★ 
MARFORCOM^r  ★  k 
NAVI  FOR  ★  ★★ 
SPECWAR  ★★ 
NECC  ^ 

NWDC  ★ 


ONR  Global  Prague 

Science  Director 


Bahrain 

/  NAVCENT/C5F  ★  ★★ 


Naples 

NAVAF/NAVEUR/ 

C6F  ik  k  k 


“N 


India 


ONR  Global  Tokyo 

Science  Directors 


III 


O  ONR  Global  Sao  Paulo 

Science  Directors 


ONR  Global  Singapore 

Sceince  Directors 


O  Science  Directors 

•  London 
•  Prague 
•  Tokyo 
•  Santiago 
•  Singapore 
•  Sao  Paulo 


ONR’s  Global  Offices  are  the  Bridge  to  International  Partnership; 
Naval  R&D  Diplomacy  in  More  than  60  Countries 
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Contested  Urban  Environment 


Area  Description 

Technical  Approach 

Complex  terrains: 

•  Urban  fires  and  weapons 

•  Crowded  and  cluttered  physical,  human,  communication, 

•  Urban  mobility 

and  informational  environment 

•  Urban  communication 

•  Physical  compartmentalization  and  additional  dimensions 

•  Threat  sensing,  detection,  and 

•  Proliferation  of  observation  and  fires  technologies 

prevention 

•  Threat  obscuration 

•  Urban  survivability 
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Cooperative  Autonomy 


Area  Description 

Technical  Approach 

•  Extend  reach,  increase  mass  and  quantity,  and 
augment  the  capability  of  expeditionary  forces 

•  Allow  penetration  of  environments  too  dangerous 
for  manned  systems 

•  Retain  capability  despite  combat  losses  with 
automatic  and  flexible  unmanned  adjustment 

•  Disperse  capabilities  associated  with  traditional 
capital  assets 

•  Low  cost  ground,  air,  and  amphibious 
autonomous  systems 

•  Distributed,  collaborative,  coordinated  and 
cognitive  autonomy 

•  Autonomous  sensing,  obstacle  detection 
and  path  planning 

•  Unmanned  C4  and  control  theory 

•  Manned-unmanned  teaming 
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Undersea  Autonomous  Systems 


Goals 

•  Mobile  autonomous  environmental  sensing 

•  Predictive  capabilities 

•  Adapt  systems  to  environmental  variability 


20 
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Autonomous  Surface  Vehicles 


Goals 

•  Perform  complex  tasks  in  a  complex  environment,  without 
human  intervention 

•  Respond  effectively  to  dynamic  situations 

•  Perceive  environment,  internal  and  external 
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Aerial  Autonomous  Systems 


Goals 

•  Safe  operation  in  the  maritime/shipboard  environment 

•  Effective  collaboration  with  humans 

•  Increased  role  with  greatly  reduced  need  for  human 
intervention 
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Expeditionary  Autonomy 


Goals 

•  Affordable  platform  agnostic  modular  Autonomy  kits  for 
current  and  future  fleet  vehicles 

•  Seamless  &  natural  visual  and  verbal  human-robot 
interaction 

•  Multi-platform  collaboration 
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Novel  Systems 


•  Multi  domain  platforms 

•  Fundamental  understanding  of  the  hydrodynamics  of  high 
efficiency  bio-inspired  underwater  propulsion 

•  Compact,  low-power  perception  and  mapping  for  nano-UAVs 

•  Muscle-like  actuators  and  multifunction  material  control  surfaces 
(undersea  and  air) 
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Cognitive  Advantage  and  Artificial 

Intelligence 


Area  Description 

Technical  Approach 

•  Meld  machine  intelligence  and  human  decision  making 

•  Data  science  and  analytics 

Ground  warrior  advanced  decision  support 

•  Image  classification 

•  Enhance  warfighter  sensing,  cognitive  speed,  and 

•On-board  processing 

decision  superiority 

•Augmented  Reality 

•True,  rapid,  all-source  data  fusion 

•Visual  attention  models 

•  Knowledge  products  delivered  to  the  warfighter  with 

real  world  context 
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Expeditionary  Communications  and 

Cyber 


Area  Description  1 

1  Technical  Approach 

•  Resilient,  robust,  and  secure  communications 
•Cyber  and  information  warfare  capability 

•  Electromagnetic  signature  control  and  influence 

•  Rapidly  changing  network  conditions  amidst 
battle  of  signatures  and  physical  movement 

•  Exploits  close  physical  proximity  while 
mitigating  connectivity  shortfalls 

•Networked  and  local  computational 
availability 

•  Non-GPS  precision,  navigation,  timing 
•Antennas  and  propagation 
•Communications  and  information 
theory 

•Communications  signal  processing 
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Signature  Visualization 


2/6  S-2  in  Bn  COC  at  FEX  II  (Dec  2016) 


USC  ICT  Aerial  Terrain  Line  of  sight 
Analysis  System 

https://www.youtube.com/watch?v=-spEV8dkuOY 
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Opportunities 


Advancing  Artificial 
Intelligence  for  the 

ONR  BAA  Announcement  #  NOQOI4-18-S-B0O2 

Naval  Domain  - 
22  March 

BROAD  AGENCY  ANNOUNCEMENT  (BAA) 


Armored  Reconnaissance  Vehicle  (ARV) 


Advanced  Technology  Development 


Future  Naval  (  apahilits  (F 

NC) 

0j\R  Special  Notice  X00014-18-R-SX05 

Special  Notice  N00014-18-R-SN05 

Special  Program  Announcement  for  2018  Office  of  Naval  Research 

Basic  Research  Opportunity  : 

'‘Advancing  Artificial  Intelligence  for  the  Naval  Domain” 

I.  INTRODUCTION 

This  announcement  describes  a  research  thrust  entitled  "Advancing  Artificial  Intelligence  for 
the  Naval  Domain"  to  be  launched  under  the  Fiscal  Year  (FY)  18  Long  Range  Broad 

Agency  Announcement  (BAA)  for  Navy  and  Marine  Corps  Science  and  Technology. 

N00014- 1 8-S-BOO 1 ,  which  can  be  found  at  https:  www.onr.naw.mil  en  Contracts- 
Grants  Fundins-Opportunities  Broad-Asencv-Announcements 

The  research  opportunity  described  in  this  announcement  falls  under  the  following  sections 
of  the  BAA:  Appendix  1  "Program  Description" 

•  Section  I  entitled  “Expeditionary7  Maneuver  Warfare  &  Combating  Terrorism  (Code 

30):  specific  thrusts  and  focused  research  area: 

•  Paragraph  E.  "ONR  30  Decision  Support.  AL  Machine  Learning  and 

Armored 
Reconnaissance 
Vehicle.  Full 
Proposals- 

April  2 

Long  Range  BAA 
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Staying  In  Touch 


The  Office  of  Naval  Research  (ONR)  coordinates,  executes,  and  promotes  the  science  and  technology 
programs  of  the  United  States  Navy  and  Marine  Corps. 


Command.  Control 
Communlcafcns, 
Computers;  Intelligence. 
Survellanoe  and 
Reconnaissance  (C4ISR) 


Warfighter  Performance 


W)  #  https://www.onr.navy.mil/  P  -  A  C  Office  of  Naval  Research  H...  X 


EKT5 


Technology  Locator  Careers 


Bsa 


About  ONR  Organization  Our  Research  Contracts  &  Grants  Education  &  Outreach  News  WbrkVWthUs 


Directorates 


ONR's  directorates  balance  a  robust  science 
and  technology  (S&T)  portfolio,  allocating  funds 
to  meet  the  warfighter's  requirements. 

03R  Office  of  Research  » 

03T  Office  of  Technology  » 


New  Autonomous  Flight  Technology 

Autonomy  options  for  the 
Marine  Corps  have  taken  a 
I  major  step  forward,  as  officials 
at  the  Office  of  Naval  Research 
(ONR)  announced  today  3 
successful  final  helicopter  flight 
demonstration  with  autonomous 
capability  at  Marine  Corps  Base  Ouamico.  Va„  part  of  the 
Autonomous  Aerial  Cargo.'Utility  System  (AACUS)  program. 
AACUS  is  a  partnership  between  ONR  and  technology 
company  Aurora  Flight  Sciences. 

R-=-  press  release 


Quick  Links 


Get  shortcuts  to  commonly  requested  topics 
below 

Download  the  Naval  R&D  Framework  * 

Watch  2017  Naval  S&T  Expo  presentations  * 
Watch  a  video  about  the  role  of  naval  research  * 
Read  Future  Force  magazine  > 

Explore  research  funding  opportunities  * 


Departments 


New  Mapping  Software 


US.  NAVAL 

LresearckJ 

LABORATORY 


ONR  Networks  Around  the  Globe 


30  Expeditionary  Maneuver  Warfare 
and  Combating  Terrorism 

31  C4ISR 

32  Ocean  Battlespace  Sensing 

33  Sea  Warfare  and  Weapons 

34  Warfighter  Performance 

35  Naval  Air  Warfare  and  Weapons 


informatiorvcyber  domain. 
Rsad  cresi  release 


Science  advisors  from  the 
Office  of  Naval  Research  Global 
play  a  key  role  in  today's  naval 
technology— connecting  the 
warfighter  to  cutting-edge 
capabilities  in  the  air.  at  sea.  on 
land  and  in  the 


:::H  r 

* 

«...  • 

www. onr.navy.mil 


flickr 


29 


This  Slide  is  Distribution  Statement  A:  Approved  for  public  release.  Distribution  is  unlimited.  DCN#  43-2983-17 


